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ABSTRACT < , 

The paper extend? our earliier analysis o*f teaching to the analysis 
of the best teachers' for whom we could bbtlin transcripts. The 
analysis ^attempts to specify the set of goals and subgoals that 
guide 'the teachers^, the- s^ of, specif ic strategies that the teachers 
use to generate cas.es, questions, and comments for the. students, and 
the control, structure that the teachers, use to allocate their time 
between different goals. The theory constructed from this analysis 
can be af^lied in educating ,j:eachers to be effective, and in 
building intelligent, CAI systems' of the futCire. 
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INTRODUCTION • ' • 

■ : ■ ) 

In previous work (Collins, 1977; Collins, Warnock, Aiello, and 

* ■ . ■ 

Miller, 1975a; Collins, Warnock and Passafiume, l'975b; SteVens and 
Collins, 1977) we have attempted to build formal process theories qf 
the goals and strategies of human tutors. In thi^s paper we attempt 
to analyze the strategies of the very best teachers' for which we 
could obtain films or transcripts. ' • - • 

The^ teachers we analyzed have diverse teaching goals^and 
strategies. Nevertheless we can abstract out common -elements in 
their teaching ,'• as weLl as reasons for the differendes. All of the 
teachers use^some version of the case, inquiry^ or discovery^ method 
of teaching (Anderson and Faust; 1974; S igel ' and Saunder s , 1979). 
They do not simply teach facts>^,but rather* they teach basic 
principles or basic problem solving strategiies for approaching 
different kinds of problems. For example, one teacher- we analyzed, 
is particularly effective in teaching his students how to attack 
problems. His students end up using many of the^. same techniques he 
uses to approach novel problem^. Such an outcome indicates that it 
possible to teach problem .solving strategies and that these 
techniques are sufficient to do so . ^ " • 

^ The theory of interactive teaching that we arie cons tr uc4:ing is. 
ca^.t in' a framework simil/ar to that used by Newell and Simon (1972) 
to describe human problem solving. It contains three parts: 



1. The "goals and subgoals of effective t^cK^^s. ' ^' ' 

•'•2. The strategies used to realize (different go^ls and ' 

J 

subgoals. • 

. , 3^ The control structure for selecting" and pursuing 
different goals and subgoals. ' 

» 

Teachers • typically pursue several goals simultaneously. Each. goal 
has associated with it. a set of strategies for selecting cases, 
asking questions, and giving comments. These are represented in our 
theory. as condition-fiction pairs (Collins, 1977). In pursuing goals 
simultaneously, teachers maintain an agenda (Collins, et al., 1975b; 
Stevens and Collins, 1977) which allows them to allocate their 'time 
among, the various goals efficiently. -i-he theory therefore 
encompasses goals, strategies, ajid control structure. 

We , see two kinds " of uses for ? formal theory»of interactive 
teaching." Cui-rentiy there is much active research , to ^'deve lop 
intelligent computer assisted instruction (ICAi) systems (e.g., 
Sleeman and Brown, 1979; Goldstein andBrown, 1979). To the degree 
*we can develop precise theories of effective teaching strategies, 
these can be embedded in ICAI system^. Equally important are the 
imprications 'for teacher education. We think we can makei explicit 
the kind^ of goals our best teachers pursue, and the* specific' 
strategies they us^ for dealing with different kinds of situations. 
In summary,, we think it is possible to^make the accumulated tacit 
knowledge of ou-r best teachers explicit enough both for future 
teacKers to learn and for ICAI systems to use.. 



TerminolV^y used in the -theory , y 
\ Many of the * teacMng strategies ; 'we " describe serve to 

communicate 'the teach^^'s understanding of ' the causal structure of .a 
^omain to a student. Thus , we ne^d^ a way to notate a caudal 
st-ructure. One way of " represent in^' icau'sal dependencies is in terms 
of 3" ai>d/or graph (Stevens "and CoJLlins, 1'980) . ' Figure 1 shows su9h 
a^ graph for the caosal dependencies derived by a student in a 
dialogue that one of as conducted on growing grain in different 
places (Collins, „ ef al., 1975^) . Each place that was discussed 
funcVioned as a case in the terminology of ' the theory. In the 
figure rice grov^ing is the dependent variable , and is treated as a 
function having two possible valges ; either you can grow rice or- you 
can^t. In other sections of the dialogue wheat growing and corn 
growing wer e^3iscussed as alternative dependent variables. lSl|ili>e 



grain gi^cfwing7^'\*h ich the student treated as a threshold function, 
many dependent variables are treated as continuous functions (e.g.- a 
place^ IS colder or warmer), where there is a continuous range of 

' * * 

values^. * . ' . .. 

During the course of the ;dialogue the student identified four 
principal factors affecting rice growing: fresh water, a' flat area, 
fertile soil, and warm temper at^n-e . These were configured as shown 
in the, diagram. .These factors (or independent va'r iabj.§S ) . are linked 
to rice growing tftiripugh chains with various intermediate g^eps . In^ 
fact any 'node in a chain can be considered a$ a factor affecting" 
subsequent nodes. Figure 1 itself represents 1; only -A Vtop-leveT'" 
description, since nodes or links in the diagram can be eKpand^d tol 



^ 



more detail (Stevens ^ Collins, 1977). Links expand . into chains 
of links and nodes, so that for example "irrigation" can be 
considered an intermediate node on the chain from "river or\Xake" to 
"supply of fresh water". . ' 

_ Given' ^a set. of ' factors, and a dependent- variable, 'a rule (or 
hypbthesig) is any function that relates ^values of one or more. 
^factoi?s to values of the ; dependent ^ var iab.le .• A rule can be more or 
less complete depending on the degree it takes into account all the 
relevant factors and the entire range of valifes of the dependent 
variable. For example a- rule about:rice growing might assert that 
growing rice depends on heavy rainfall and fertile soil./ Such a 
rule is obviously incomplete with respecjt ,to the mini-theory shown 
in Figure 1. A theory specifies the causal structure "interrelating 
different rules. in complex domains like rice growing and medicine, 
no theory is ever complete. 



Insert Figure L here " 



\ 

t 

Given the dependencies in' the diagram^ it is apparent- that a 



'A 



factor like heavy rainfall is neither necessary, nor sufficient for- 
|jice growing.' ^t is not necessary because obtaining a supply of 
tre,sh w^tfer (which i$ a nece.ssary . factor ) ^can also be' satisfied by 
irrigation > from a rivet ' or lake. It is not "bufficient because 
other factors, such as ^ wa'rra tel^perature, are' required . When pfior 
'node^s are ^.conjiected into a node by 'an "or"^.any of the prior" nodes 





til 



Figure 1.^ A student's analysis' of the causal factors 
affecting rice growing. 
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is sufficient and none is' necessary .-.with -respect to that node.- For' 

• ♦ • » ^ _ > * 

. examjJle, either heavy rainfall or a river or a lake is a' sufficient 
_soi^rce for .£4;esh water^ biit none of these .is necessary for fresh 
water. In contrast when prior* nodes are connected into a node by 
an "and'*^, all of , the* prior nodes are .• necessary and 'none, is 
sufficient with respect' to that riode, For example, fresh wateV is 
necessary to flood a flat- area, ^but is not sufficient; Though heavy 
rainfall is 'sufficient as a source of fresh water, it is not 

* , , ' 

sufficient for growing rice,- because of the "ands'" in the' causal 
structure between rainfall and ripe. Any variable not included as a 

* • ^ ■ ■■ " 

factor in the diagram is- effective!/ . treated as irrelevant to . the 

theory • ^ ^ 

» \. ■ * 

Independent and Dependent Variables in 'Different Domain s 

Table 1 illustrates how the' terminology applies to teaching 

strategies in different domains We believe that these* teaching 

techniques can be' applied to' vir tually' any domain. In Table .1 we 

ar^x -mot trying vtd ii$t all possible independent and dependent- 

\ ■ . . . .. : »j ' -■. . • . ^ . • ■ 

vai;iabies, nor .are we ruling out other possible assignments; these 
are tHerely meant* to indicate the most common assignments that 
teacher's make, ' .' 



Insert Table 1 here 



Lj?t us briefly explain these examples:. . 
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^ t . . • 

» , ■ . ■> • ^ 

- . . : ■ ■ • • 

In acJthmfetic) "a'-StUdentVsolves problems 'in " ocder ,t(^(, 

}.earn how to. handle di'ff^rent opera-tiops, nuinberf^, 

variables, etc.; B^ca'Ais^s' of- the procedural .empjias*ife • i/i 

arithmetic it is. 'the domain that . fits' our terminology 

least well . . ' 

In art history, the teacher attfempts to' teach $t\idents 
how .techniques, uses of textute St color, ^tru(^ural 
interrelationships, etc., oi^eate certain effects on th6 
viewer . ' 
In law, hisbarical cases are used to teach students how 
different variables (historical precedents, laws; 
aspects of the . ; par t icular case; ,etc^) affect legal 
outcomies . - ^ i . . * 

In medicine, the goal is to teach students how to 
diagnose different diseases, given patterni^ of 
symptoms, their course of development, and the 
patient's history and appearance. \ 
In geographj^K^ most variables ^are treated both as 
independent and d^endent variables on different 
occasions/ Ftfr example, 'average -i^emper ature is a 
dependent variable with respect to the firsttbrder^ 
factqrs, latitude an^ altitude/ -and- . general 
second-order factors, distance f rbm the sea, wind and 
sea currents, tree and cloud cover, etc. But, in turn, 

0 

temperature is a factor affecting dependent variables 
such as population density, prdtlucts, land types, etc. 



,In moral edyca€io.n, tefacherft) try to t^h rules of 
moral behavior by considering difjEerent sikuatigns wi^h ' ' 

respei^vt .to the -actions and motives of the participants'., 

. ' ■ • ' " s \ 

• 7, In botany, one Yearns what ponf igM^rations of the shape, 

branches^ leaves,- ^tc, go with what tree and plant 
names . /""n 

••• • J ■ ■ ,■. , . • 

Whether a variable is treated as a dependent "or independent 
variable defends on wha^ the teacher ija trying ^o teach. It does 
not depend on the directio/n of causality. For- example, in geography 
or law, the independent variables (e.g., amount of available water) 
are typically presented as causes of the dependent variabiles (e.g., 
population, density) . On the other hand, in electronics or medicine 
where the emphasis Is on diagnosis, the independent variables (i.e.,' 
symptoms) are typically effects caused by the dependent variables 
(i.e., faults or difSeases^. In some domains, such as botany, thfere 
is no pausaiity in. either, direction. ^What functions as a dependent 
variable is ^merely what one, tries .to make . predictions ' about in the 
real world. * • 

ir 

• " I ■ ' 

. ' DATA ANALYZEID 

V Th^^dialogues we have analy;zed range over a variety of domains 
and situations. Some ar"e with individual students and some with 
groups of, students. The students range in ag^ from preschoolers to 
.adults. In. some case's the teacher has a well-worked out plarf as to 
yhere the dialogue Will go, whereas in others the teacher do|s not. 
We qaYi illustrate/ the variety by describing briefly e^ch of the 
dialogues we have analyzed. ' ' , 
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Tabre^^2 lists the "dialogues we have^ analyzed mo^t recently, 
f he Me no dia;ogue by ^lato'(1924) operates as a case method on ."two 
levels. / On one level Socrates tries to convince 'Meno *that all ideap 
are innate by, demonstrating that a slave boy "knows*" a square with 
sides \/T units long iS" twice as -large as a' square with SiT^e's 1^ uni€ 
long. On the'^' second level Socrates gets the slave boy to-^gure out 
the area by considering different possible Ci^nstr ucti<>ns : Socrates 
frequently uses entrapment strategies ^getting the slave boy to make 
a wi;/Crtg hypothesis, and then tracing the consequences of tKe 
hypothesis until tl?e slave boy sees the "contradiction. Socrates" 
purpose ' is not entirely pedagogical and therefore there is very 
li ttle, problem solving or discovery 'by Meno or the slave boy. 

In^fert Table 2 here 
t ■ . ~^ t 

Another transcript is from a film series showing Max Beberman, 
a famQus math teacher, teaching junior-high* students. In the 
transcript Beberman starts out by giving students problems to work 
that involve a comEllica ted protr^dure^ for computing the 'sym of real 
ni^mber^,. The' procedure involves going right on a piece of graph 
paper the distance corresponding to any positive number being added, 
and 'left ' f ot any negative number.- Students quickly start using a 
shortcut"; they add the positive numbers together, the negative 
numbers together, ^.nd take the difference. That is they learn a 
generalized procedure for adding real numbers. Later Beberman tried 
to get the students to formulate' the rulesi for addition of real 



* 



J 



\ Table 2 , 

Dialogues Analyzed with Cases, 
Independent and Dependent Variables Specified 
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1. Socrates with slave, boy ii\ Meno dialogue 

(C-square with twice areai IV- length bf sides €^;^diagonal ; 
DV-area of square) • ^ * , . 

2. Beberman with junior high ' students on addition of real numbers 

(C»problems; IV=nunjbers, directions; DV»answers) 

3. Anderson with junior high girl on. the di-str ibutive law in 

arithmetic 

(C=problems; IV=numbers, operators, order, sum;' DV=answers) 

4. ,Warman with pre-schoolers (3n who can play with blocks . 

(C'situation; IV=girls ^nd boys rights, action^, amount of 
playtime;" DV=fairness) 

5. Warman with pre-schoolers on morality of characters in Pete 

Pan • V I 

(C»characters ; IV»actions and motives of characters; 
DV-morality) 

6. Schank with graduate students on . planning .(3 phases) 

^ tC«j|jfeal wor^d goals; IV-proper ties of actiojT^'^nd " motives; 
DV«plans vs. nonplans) 

4 

(C»same; IV=same; DV«primitive types of plans) 

(C»same; '^tV-pr^imit ive 'types of plans; DV-corapiicated plan) 
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numbers as shown in this short excerpt ; (The excerpts are annotated 
with the goals and stra.tegies explained later'.) * ' ^ 



T: I want to state a rule here' wh ich would tell somebody how to 
add 'negative numbers if they didn^t "know how to dp.lt before. 
Christi^n^? (Ask for tule formulation.) 
S: The absolute value — well-^a plus b equals uh — negative — 
T: Yes, what do we do when we try to do a .problem like that? 
. Christine ^ is on the right track. (Reward rule formulation.) 
" What do you actually 'do? Go ahead, Christrhe, (Ask for rule, 
^formula^on . ) ' • * , 

S: Yon add the numbers of arithmetic 5 and 7, and then you — ' 
T: I add the numbers of arithmetic 5 and^7; but how do I get the 
numbers of arithirtetic when I'm talking with pronumerals like 
this? (Ask fbr generalization of factors.) 
S: Well, you can substitute. , . * 

\ Tf Byt I don't want to talk about any special cases now; I want 
to .talk about all the c^ses at once. (Ask for generalization 
of factors.) 

H. 

What we see Beberman doing is creating a situation where students 
working, problems will induce an abstract rule fo;: addition of rqal 
"numbers. • . ' 

The 'third transcript we analyzed shows Professor Richard,, 
Anderson of the University of Illinois teaching a junior-hi-gh girl 

•T 

to induce the distributive law in arithmetic. It parallels the 
Beberman transcript in that a series of. problems is given in" order 

' -12- 

o ^ ■ , iff ^ 
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%9 get the student to Induce a general, rule, Anderson: carefully 
selected probletns to , ipake the- distributive law apparent. Some of 
\hi3 can be seen in the for;Llowing excerpt from 'near the beginning of 
the dialogue: (.numbers are written on the board) 

Tj^ OK. Clqse yout eyes..?'^ 6 +' 6 = .. Now. (Pick 

differentiation exemplar for first factor. Ask f or* pred iction 
of dependent variable.) " . 

S: 60. 

• T: OK. Close your eyes. 7 x 12 + 3 x 12 = .., Open 'em up. 
(Pick differentiation exemplar- for first factor. Ask for 
predict^ion of dependent variable.) - , 

S: (Goes off to side to work each part.) 

T: Emmm. - . . / 

■I * 
S: No? • • , • 

• 

T: 'Thatje/s all right.' You can do that. But you're still doing it 

L' 

V ' . 

the hard way, Margie. (Punish lack'of rule formulation.) 
S: 120. 

T: > Now look' for a second at the problems that are up. on the 
. ^oatd. Don^t ^say anything. But just look afit^ all of the 



problems - and the answers and see if you see any th ing 

^ interesting. Don't tell me if you do, but just look. 

i 

(Suppress rule verbalization.) Loor->at the, problems and the 
arrswers. (Ask for rule formula t iory. ) 

I** » 

/nderson picked his cases so that the pattern Was obvious; the 

numbers that are added sum to 10 so that the multiplicati\/e^ factor 

(6 in the first case, 12 in the second) shows up as the significant 



^ • t ■ • ' ' ' 

digits in^ the answer. Anderson l^ter varies the par^ticular digits 

:{e 4 instead of 7 .+ 3) and thea their sum as. well, until tf»e.-_ 
student formulates tlie distributive law in its most 'general form. ' 
The dialogue mainly illustrates 'how case selection can be used to 
force generalization.^ " . ^ • 

r 

The two transcripts with Eloise Warman show how similar 

techniques can be applied to teaching moral education. Warman in 

the first dialogue tries to get a group of preschoolers to Kvrmu[La\j:e 

a new rule in the classroom for allocating the toy ^blocks to tl>e 

boys (B.) and girls (G) . In the second dialogue she tries to get the 

children to evaluate the morality of the different characters in the 

play, Peter Pan, which they just saw. Two excerpts, one from the 

very beginning of the»fifst dialogue (Whidh states the problem) and 

one from near the end (which gives the new rule), illustrate^ her 

V 

techniques: 

_ T: T\e problem is that the girls say the bo^s never let them play 
a with bXgcks. (Estab-lish rule formulation goal. - Subgoal .have 
" kids formulate rule. Point out insufficiency of factors in 
current rule.) But^What do you think ^bout this boys,^ that 
the girls play with legos and you can't play with legos? 
(Hypathet ical case construction ^or insufficient factors. .Ask 
if rule is correct or incorcect - i.e., f^ir or not.) 



G: I think it should be the teachers. 
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T: Pu\: why just, -tite teach.ers? (Xgk for questioning' of 
authority.) > ];t doesn't seem , to work, 'we had >j^n,idea. /We^ve 

been trying., /(^oint out insufficiency of factoxs in rule.) ' 
,/ • 

«. 

B: I've got one id*a. ' ^ a 

T': Qh, Gregg's .got a good idea. (Reward rule forinulatidn . ) 

B: The, p iris can •pi.ay^with the big blocks only on 2- days. 

^ T: Hey, listen we come to school 4 days a week. If. the girls » 

play- with the big blocks' on 2 days that givers the boys 2 otheV 
. ✓ ' ' . ' ■'• 

• days *• to,, play with blocks. Does that sound fair? (^Restate , 

^ rule.^Ask ;f rule is correct or incorrect - i.e., fair or 

not.) , . /\ 

G: Yeal Yea 1 " 4. ' - . 

•The initial se.gment shows Warman's .statement of her overall goal to « 
formulate a new rule for allocating • blocks She then suggests the 
kind of hypothetical case she uses frequently where she proposes ^ 
situation with the opppsi te^ value as to who gets the advantage, and 
asks the group who had the advantage whether they think that is 
fair. The seTOnd seghent illustrates how she encourages kids to 
verbali*ze their ideas and to question authority; , She even manages 
to entice a fairer rule out of one of the boys in the class. ' 

■ Y ' ' ^ • • ■ 

Andther transcript we analyzed is based on a class in 
Artificial Intelligence taught by Professor Roger Schank at Yale, 
^here were three phases to the class session: in the first phase the 
goal was to specify what fe^ures define a plan; in- the second phase 
the goal was to defin'e a tax9nomy of basic- plan types with respect 
to the definition in the first phase/ in the third phase tfie goal 
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w*i.^analyze a real world plan in; terms of the taxonomy. The 
.excerptX beldw frbm the ffrst^phase illustrates the establishment of 
fbe ini^Ul goal; and two of the definitions (i.e., .rules in our 
terma>-4ormulated by the students: * 

• V ■ • 

T: It's not an unreasonable' quest ion' to ask how hew plans might 
arise, but it's not. the right time to ask it. , The first 
thing to ask is what's'.a plan?' (EstabUsh goal: Identify 
factors • necessary toz something to be a plan. Strategy: Ask 
for rule formulation,) 

SI: A means for getting to some desired state. 
; T: Anyon^ else? (Ask if rule is correct or incorrect 'or for 
formulation of another rule.) - ' 

S2t/The^y're heuristics which people learrAo use" to deal with 
certain types of situations. 

two excejrpts '^below from the second phase illustrate the 
establi^ment of the goal characterizing different, plan types and 

on4 of Schan\'s many attempts to get/the students to question 

r 

authority. m this phase the student's taskNis to form a disjoint 
set of basic plan types: in^our terms this is A set of sufficient 
factors joined by "or" links. ' . 

T: I'm going to make you classify again. What kinds of plans 
are there? (Establish goal: identify possible plan types. 
Strategy: Ask for sufficien.t factors.) Unless you would 
like to change the definition the only thing I'm goin^ to let 
you answer is types of means. (Point out irrelevant 
factors.) . ' 



S2: Plans to establish abcial control over something. ' ' > 

T: ' q;he two Jl^ou ate^greei^g that everything fLm the book ^ is 

> . • ' , I*- • 

gbspel. j(56int out' factors are same as authority's.) ; It's 

all right." "^ive me something new — I wrot^ those ~ ,lnventf 

something^. (^sk for , questioning of Authority. Ask for 

sufficient factors.) / •. ' 

( / * ■ 

sa: Plans to establish conversational offensive over ope's 

• ■ / ^ ' 

dialogue . y , * . w. - 




T: Let's- just call it- cofiyir sation , plans. (-Restate factor . ) 

OK? What^else? (Ask for sufficient factors'.) 
S2: How about plan^\tb manipulate objects?, ^ ' - 

SI: Plans to manipul*cree people. 



Many tof the indiv.idual strategies in the Schank dialogue, such as 
encouraging verbalization and quest ioningi of authority, parallel the 
strategies in the Warman (^ialogj^s. This is: probably because both 
place heavy emphasis on teaching the students to f6rmulate their own 
rules or theories for dealing with novel problems. 

Table 3 lists the dialogues that we analyzed in.earlier papers 
^ (Collins, 1977; Stevens and Collins, 1977). 'These ranged^ across 
, geography, medicine, moral education, and letter identi|ioation. 
• The theory presented i/ere incorporates the goals ^^and strategies 
identified in these earlier analyses, though in some cases the names 
of specific strategies' have been changed to fit the more general 
structure imposeid upon the theory in this pAper". * 

* • *17- 
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^ TI^E THEORY 



Goals of Tea*chers 



There is a jirogreasion shown in, Table 4 among thfr goals that 

0 • * 

^different tea\:hers pursue. ; T^e first most basic goal is fio teach 
students the facts and concepts that comprise, a 'domain t)f knowledge. 

• ' , ■ . y - 

We-arialy^ed dialogue-s df this^kind in an earlier p^per (^ollfns, et 
al. 1975b), but none of then dialogues discussed in this p^per are of 
,this kind. A second higher level goal is to teach students a* 
particular rule or theory underlying a^ndomain.pf facts and concepts-. 
This kind of goal was evident in the Beberman,' Anderson-, Socrates> 
Stevens >^nd Collins, and Swets and Feurzeig dialogues'. The- thitd 
and highest-level goa> is to teach students how to. derive a - new 
theory for a domain of knowledge. This differs, from the second goal 
in that the teacher has no a? priori expectat|Jon of what^the 
to-be-d^rived theory is, rather the teacher has an idea about what 
constraints -the theory must satisfy. This kind of goal is evident 
in the Warman and Schank dialogues. ^ > 



. , Insert Table 4 here 
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Table 3* > '■ ^ 

Dialogues Analyzed Earlier with Cases, • 
Independent ^nd Dependent Var i'ables 'Specif ied ' 

Xnderson with, hypothetical college - s :udent . on factors 

affecting temperature ' . " 

■ > ••■ ■ • , 

(C^places; ivflatitude, distance from sea? DV=teraperature) 
Anderson with- hypothetical college student on ..raorality of 
draft resistance * , , 

(C=draft" resistors arid American d^voIu tionar ies ; IV=acts, 

motives; DV=morality) 
Stevens and Collins with secretaries and high school students 

on factors affecting- rainfall , . 

(C=places; IV=currents, evaporation, cooling; DV=rainfall) 
Collins with secretary and^ientist on where different grains 

are grown . - . / ■■ . ' 

(C=places; iv=climate, soil, water, terrain; DV=rice, wheat) 
Collins with secretary »and scientist on population density 
.(C=places; \ IV=cliraat€, products, ' transportation; 

DV»population density) ' ^ ' 

Swets and Feurzeig • with hypothetical medical student on 
' diagnosing disease 

(C»medical case; IV=symp toms', history; DV=disease) 

Swets and Feurzeig with hypothetical student on fdetitifying 
letters 

(C»lQtters; IV«letter ftatures; DV=letter names) 



Table 4 • . 

• 0 * 

. , Goals of Te«chej:s 

Teach facts and conc^p'ts about a domain. . « ' 

Teach a 'particular rule or theory for a domain. 

a. Derive correct rule or theory. • 

b. Debug iricprrect rules or theories. 

c. Learn to make predictions from the rrule or theory. 
Teach how to derive d rule or theory ^for a domain. 

a. Learn what questions to ask to construct a ruleor 
theory. ' ; 

b. Learn the nature of* a rule or theory. 

c. Learn how to test a rule or theory. ■ 



. The goal- of teaching a t)articular rule or theory'has associated 
" . . \" ■ ■ ' " ' ■' 

with it three basic subgoals: ^ ^ , 

• ^ ^ . » "* 

«• ' , 

♦ ^' ■ ■ ■ ■■ ■ ■ ' ■ 

' . . J.) The major s'ubgoal* is for ^thfe . student 'ta ' analyze different 
cases ir, orde?^ to' der ivp the rule or thiory that-the teacher 
has in^ mind. For example, in arithmetic Bebermart tried- to 
get students to derive the rule for. addition'' of real 
numbers, and Anderson the distributive law. ., In geography 
'Anderson tried to get the student "to understand how 
distance-f rom-ocean affec^ts temperature, and Stevens and 
Collins tried to get students to build a first-order theopry 
of the factor!? affecting rainfall. The casg selection, 
questioning, and commenting strategies shown later in.Tables 
5 and 9, are the principal strategies used to teach' a 
particular theory. • ' 

2) Along with trying to teach a particular rule or' theory, 
teachers often try to elicit and "debjjg" incorrect rules or - 
theories. Thfe teachers want" the student to 'confront 
incorrect hypotheses during .learning, ^so that they^ won't 
fall into the same traps later. ' This kind of goal is 
evident in Soctates' dialogues where he often traces tHe 
consequences of his student'^ 'hypothesis ^ down .to a" 
-contradiction, and in Anderson's dialogues on geography and 
' moral education where he" entrjsfps, students into revealing 
their misconceptiQns . ' jhe entrapment, the counterexample, 
and the hypothetical case .construction strategies shown in . 
Tables 5 and 7 are particularly important to debugging . 
incor rect, hypotheses.' 



3) Another goal that frequently pairs wi^^b teaching a^ given 
. rule.s^or theory is teaching how to make- novel predictions 
, ^ based on the ru-le or theory. Simply knowing^ tfie structure 
'of ~* theory is not enough ; one must be able to operaVe on 
the structure to deal with new problems. - For example, 
\. ' Anderson in mathematics gives harder and harder problems for 
the student to .predict the answer, Stevens artd tollins in 
geography' start- with, casesv^ exemplify first-order 

factors and gradually -move to more difficult cases to 
predict. War man trie s»/'t6^^^^.:^^^g^^^ her. . prescfioolers to predict 
what will happen under different rules, an^ Swets and 
Feurzeig are. trying to, get students" to,d.iagnose noVel cases. 
The case selection >. strategies am? prediction strategies 
shown in -Table? 5 and 9 '^re crucial, to teaching students Tow 
to deal with new cases'. ; . ' '. . , , ^ ' .. 

1 When teachers, try to tfeach stu^ent^ how tp-.derive* a novel 
theory-, there are again three kinds of. siibgoals that come into play: 

/» The most important subgoal; is ■ to teach .students what 

questions to ask ip order. • derive a new- rule or ..theory 'on 

. t^eir own; For example , Warman ' teaches ■ her ^ students to 
^ M ■■ . ■ ■'. ■ • . ■ . • ' 

evaluatfte oany xule by how fair ' it As, Scha.nk is trying to get 
' ■ ■ ■ -v o • . ^ — ':■ . •;. '■ . • •' • ■' 

students to cons.tr uct a theory by. asking'par'ticular kinds of, 

questions \n a specific, order,- aiid" Swets; and Feurzeig in 

medical diagnosis emphasize considering different diagnoses 

before reaohinV^:^^- conclu^;ion. The suggestion strategies 

- -and the rt^ie,^ evaluation strategics in Table 9 are 

par ticuTaT:*iy 'relevant to these goals. 
' ■ . ' " -20- 



2) A gecond subgoal that probably underlies ipany of the 
dialbg-ues, but . which is tjiost evident in Schank's dialogue, 
is. to teacb students what foi^m a rule or' theory, should take. 
• In Schank's case, the structure of a theory is a set of 
primitive elements, serving ^ . role similar to the basi^ 

• ■ • ' « 

elements ,in chemistry. Beberman "in the segment shown was 
teaching students the form of arithmetic rules, where 
variables replace numbers in order to be gene-ral. Stevens' 
^ a,nd Collins' notion of, a tl^eory" of rainfall was a 
XiS^archically-organized> process theory. * Individual 
strategies seem 'to be only rarely tied to Jthis-goal; 
rather, the principal method for obtain l<ng .this goal is to 
get the students to construct different rules or theories of 
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the idealized type. 
3) Occasionally in the dialogues the teachers pursue a- goal of 
y teaching students how to evaluate a., rule or^ theory that has 
been constructed. For ^example, Anderson in teaching about 
the factors affecting tempe'c^a ture tried to get the student- 
to learn how to control' one factor while testing for 
another. . Schank, after hi:s students h^d specified a set of 
primitive plan types, tried to get them to^est their theory, 
^y applying it to a real world pl^ (i*e.,' becoming 
pj:esi<36nt) . The strategies teachers ^se are specific to the 
ki;nd of evaluation methods being taught. * > 

Finally, it ''was a clear goa;L of botjj Warman and Schank to get 
their students to verbalize and defend their rules or theories. 
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This goal is . clearly at . a different level lErom the other kinds of 
goals described-, and operates concurrently with the other top-level 
goals. For example, it is clear why Warman's children were always 
interrupting to give their ideas: she was constantly encouraging 
and, rewarding them^for joining in. Similar ly,^ Schank tries to get 
each student in the class to either offer their own ideas, adopt 'On? 
of the other's ideas, criticize one of the other's ideas, etc. Both 
stress the questioning of authority in their dialogues as a means to 
push students to formulate their own ideas. The strategies for 
encouraging verbalization in Table 9 serve this goal. 

ft 

These are the top-level goals and subgoal'S we have been able to 
identify ^ so far. In pursuing these goals, \ teachers adopt 
lower-level- subgoals of identifying particular oijrtissions . or 
misconceptions and debugging them (Stevens' and Collins, 197T) . Thus 
these top-level goals spawn lower-level subgoals that drive the 
dialogue more locally. This will be discussed' more" fully in 'the 
sectiprf^on control structure. 

.> gtratfeg'ies for Selecti rl^ Cases, Questions and Comments - 

f , . — 1 

-.. I^opking^at the- fine ^^truc.tuce of the dialogues one isees 

# • ■ \ * ■ ■ <- 

recurring^ patterns »of " strategies in selecting cases, asking 

' ■ ■ " k \ ' 
^.qtie-sfcioos, and making pommeftt,ff\ WA hay.e tified- to characterize the 

• ' * ■ ( \ ■ 

., individual .stra teg ies^ th^t occur irr^fej^rms of 6ondit ion-action pAirs 
„t)p projiuctipns . ((jolllns, 1977 ; \'New9 l^\F\d ^Simphr 1972^^7^^ so, 
specify the 'conditions that trigger each\str^4^egy to be invoked, 
and . the' action^ tHfat the teacher takes when the strategy is invoked. 



When the action taken is to ask a question or make a comment, 
the surface for^ of the question or. comment can vary quite widfely. 
For example, in asking f6r prior factors a teacher mig,ht say: "Why 
d5 they grow rice in Louisiana?"' or "What makes it possible to grow 
rice there?", or simply "Why?", depending on the context. Despite 
the large number of pQssible surface fprms, at a deeper, level a 
single questioning strategy Ig' being applied. 

The condition^tion pairs for each of- the str^tegie^s, together 
with examples of each «re shown in the Appendix, it is impossible 
to get 'a feel for the theory without' wading through some portion of 

the Appendix. * ' 

« 

Case Selection- Strategies ; Much of ^ the art of , effective 
teaching centers around the selection ^ the best cases. „ By looking 
at the var iousjiialpgues, we have formed an initial .theory about the 

principles governing t^achers^ selection of cases. 

, -v. 

- Table 9^ shows the different types of case selection strategies. 

There are four basic types: picking positive and negative exemplars 

for particular factors, picking comparison cases with respect to 

previous cases selected,, picking counterexamples, and constructing 
« 

hypothetical cases for particular types of misconceptions. We will 

; ^ 

describe each of the strategies v^ith reference to an example. 
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Table 5 

Different Types of Case Selection Strategies 

* . *■ ' 

Poaitive and Negative Exemplars 

Pick a positive exemplar for a set of factors 

Pi.ck a negative exemplar for a set of factors 

Pick a positive exemplar for a sufficient factor (a near hit) 

Pick a negative exemplar for a necessary factor (a near miss) 

Comparison Cases , ' ^ 

Pick a generalization exemplar for a factor (a maximal pair) 
Pick a differentiation exemplar for "a factor (a minimal pair) 
Pick an exemplar to show the variability of a factor*^ 
Pick an exemplar t9 show the variability of the dependent 

var iable . - , . ' 

Counter ex amp les * < 

Pick a counterexample for insufficient factors 

Pick a counterexample for unnecessary factors 

Pick a. counterexample for irrelevant factors 

Pick a coufltere^ample for incorrect values of factors 

Hypothetical Cases 

f f ' 

Construct a hypothetical case for insuf f ic ien.t factors 

Construct a hypothetical case for unnecessary factors * 

Construct a hypothetical case for J.r;:elevant factors 

Construct a hypothetical- case for incorrect values of factors 




i ■ ^ 

The first two case selection strategies, involve the sele.ctl6n 
.of paradigm gases to exemplify a set of factors. For example, if 
the teacher, wants to illustrate the factors that lead to different 
amounts of rainfall in different places, he or she. will stdrt' with 
"good" exemplars: either, positive like the Amazon or Oregon, v or 
negative like the Sahara Desert^ or Southern California. These aire 
good exemplars because the values on ali the relevant factors are 
consistent with the value on the dependent variable: i.e^, the 
Amazon . has all the factors that lead to. heavy rainfall, and the 
Sahara all the factors that lead to little rainfall. Cases like the 
Eastern United States are not clear-cut, and are not chosen as 

V 

paradigm case's. 

The next two strategies also involve selection of positive or 
negative exemplars, but with respect to a single factor. They are 
used ,if the teacher wants to focus the student bh a particular 
factor n9t yet identified. if the factor is' sufficient fi.e., is 
"orfed" with o/her factors) as irrigation is a suffic lent . source pf 

water for growing rice, then the teacher would choose a positive 

■ \' — 

exemplar. For example, to get a student to-«identify irrigat/on, as 
a tactor, the te*acher would choose a case like Egypt where 
irrigation is used for growing rice. If the factor is necessary 
(i.e., is "'anded" with other factor) as is warm temperature for 
growing rice, then a negative exer^plar would toe, chosen (this is the 
"near miss" strategy of Winston, 19^)3). " For example; the teacher 
might choose Alaska as an' exemplar in order to get the student to 
notice warm temperature is necessary for growing rice. Positive 



exemplars ^ are used with sufficient factors because negative 
exemplars require that all the sufficient factors be missing, and so 
do not emphasize a particular factor. Similarly, negative 'exemplars 
are used v/ith necessary factors, because positive exemplars require 
that all the necessary factors be present, and so do not emphasize a 
paBticular factor. « 

The second group of four strategies specify the selection of 
cases with respect to previous cases. These ' strategies, are best 
understood in terms of a set of factors the teacher is focussing on 
and a set c^f'dther factors (often irrelevant factors) the teacher is 
not focussing on. In picking a generalization exempllr, the teacher 
holds constant the values bf the , d^pendejit variable and of the 
factors in focuS while varying as many other factors as possible. 
For example, if the teacher hadr'chosen the Amazon .as a positive 
exemplar for the factors leading to 'heavy rainfall, as a 
generalization the teacher might pick Oregon which varies a number 
of irrelevant factor.s -(latitude, wind, direction) but holds the 

relevant factors constant. To pick a differentiation exemplar, the 

'•Vr- • 

teacher tries to hold as many of the nop-focused factor s ' constant, 

. .A 

while varying the values of focused factors and the dependent 
variable. For example, in order' to teach the distributive law 
Anderson would hold all the other variables constant while - he 
.^IpBtematically varied the number that functioned as a'multiplier 
(see p. 11) together with the answfer. This latter stijategy is the. 
minimal pair strategy used in linguistics (e.g,, Glo^Sion 1965) . in 
contrast the generi^lization strategy is a maximal pair strategy. 
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' ■ • • ' • •■■ 

The next two stratejjies are used by t-eachers to illustrate to/ 

■■ • • 

student* the- effects of varying, t^he dependent variable or the ' 
factors that are in focus, in order to sjiow the variability of a 
factor the strategy is to pick a ca^e where the dependent variable 
(and other factors) are held constant while the factor in focus 
varies. For example, to show ; the effect of "temperature on r ice 
growing, a teacher ' might pick Japan after considering JaVa, because 
they show the range of temperature over which rice is grown. To 
show the variability of the dependent variable with respect to a 
particular" factor the teacher will select a case vfhere the factor in 
focua is held cohstant while .other factors ' and ..the dependent 
v^ariable vary. For example, 'to show the variance in temperature 
near the equator, the teacher might move from the Congolese jungle 
to the peak of Kilimanjaro.' Picking cases to show the .range 'of 
variability is'' important in domains, such as medicine, where' 
students must "learn to dis^^^nguish cases that initially appear' the 
same and group cases thai initially appear different. 

There are ^ur* typ^^ of counterexample 

strategies: counterexamples for insufficient factors, 

counterexamples for unnecessary facjtors, counterexamples for 
irrelevant 'factors, and counterexamples for the wrong value of a 
factor. We can give 9'xamples f9r each of the four counterexample 
strategies. if a student says they grow rice in Louisiana because 
there is ' IoN^b of rain (which is insufficient), one can pick Oregon 
where there is lots of rain but ||b rice. If 'a student says they' 
don't grow rice in Oregon because^ it lack^a flat terrain (which .is 
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unnecessary), on^ can pick Japan which ^is also mountainous, but 
produces rice. If a student asserts rice is grown in.Japan because 

they are Oriental (which is irrelevant by the the6ry in Fig. 1), one 

' f . . \ ■ 

can pick Mongolia or Louisiana as counterexamples. a student 

asserts that a cop! climate is tieeded for-rice growing (which is the 
wrong value), one' can pick Java..and Alaska as counterexamples. 

The hypothetical case construction strategies are triggered by 
the same four situation^ as the counterexamples. If a student 
thought rice couldn't be grown in .Wyoming because it is too dry 
(which is insufficient because it. is also too cold), the teacher 
could ask "Suppose that .it rained a lot in Wyoming, do you think 
they could grow rice then?" If a student sairdTMihey grow rice in 
liouisiana because it rains a lot (which is unnecessary since they 
could use the Mississippi River for ir r igation) , ' one could ask if 
they could still grow rice if it did not rain a lot. Similar kinds 
of case?i can be constructed if the student gives an irrelevant 
factor or the wrong value for a factor by presupposing what is not 
true to be true. ' • * 
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Table 6 illustrH^^-wh^t the condition-action pair looks like 
for one, of* the Case-selection strategies: in particular, the 

< 

counterexample for insufficient" factors. There are three conditions 
where the rule might be triggered\ (1) the student proposes a rule 
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Table 6 I 

A Condition-Action Pair for a Case Selectipn Strategy 



Counterexample fory Jnauf f icient Factors 



If (1) a student , proposes a ryle or makes a prediction U^ed on 

A 

pne or more factors, that a#e insuf f icient , •\)r 
(2) is entrapped'by a rule based orf^one 'or more fact|^rs that 
are insufficient,'' • . . 

then (3)Hpi<il$ a case that has the values specif ijed on the 
insuff ici€^nt factorW^ but not the values specified on^the 
dependent variable. 

■ ■ 

EXAMPLE (from Collins on factors affecting grain growing) 

\ 

T. Why? (i.e., why do they grow rice^in Louisiana) 
^S. Places where there is a lot of water. think rice requires" 
the ability to ^lectively flood fields. 

T.^OK. Do you« think there's a lot of rice in say Washington 
and Oregon? (T selects a case where .there . is a 16t of water 
but no rice; this counterexample then led the student to 
consider t:limate and . ter rain) . ' / 



^)ased^ on, insufficient factors, (2) ma^kes a prediction bAsed dn 

insufficient factors, or (3) is en4;rapped by a rule based on 

. f ' . • , 

insufficient factors (see Table 7), If any; of these .conditions is 

met, a new case cari be selected (if there is one) that- has- the' 

conditions specified in the "then" statement. The example shown is 

from an actual dialogue on grgiin-growlng (Collins, 19^7) • 

Entrapment Strategies . Teachers use a variety of strategies to 
entrap students into revealing their misconceptions about a' domain. 
Some of _th^^ misconceptions exist prior to the teacher's inquiry, C 
but some are iri factj^rovoked by the inquiry. Bringing out the 
misconceptions alfows sjieachers to correct them directly. In this 
way tochers can act to prevent misconceptions from arising in 
future situations wh'ere the studejit is on his own. Some teachers 
shy away from using . entrapment strategies, perhaps for fear of 
forcing students .into mistakes they might not otherwise majce. But 
if there^s no stigma attached to making mistakes, then bringing 
latent misconcept ibns' into the open,, can provide a much deeper 
understanding of the domain. ^ 

♦ 

The. entrapment strateg-ies form a two dimensional space shown in 
Table 7. Xike the counterexample strategies, there are four kinds 
of entrapment with respect to (1) insufficient factors, (2) 
unnecessary factors, {3) irrelevant factors, and (4) incorrect 
values of factors. Orthogonal to this breakdown are the entrapments 
formed by proposing a rule, by proposing a prediction about the 
dependent variable, or by proposing a set of factors. We will give 
examples to illustrate t|:je different types of entrapment strategies. 
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Entrapment rules are formed when the teacher proposes a generaj. 
rule., based on some'.set ofr factors. For example, an entrapment rule 
for insufficient, factors occurs if. a student thinks they grpW rice 
in Louisiana because it rains a lot and the teacher suggests "Can 
you grow rice anywhere there is a lot of rain?" ,An entrapment rule 
for uni;)ecessary factors occurs if the teacher suggests "Do you 
always need a lot of Vain to grow rice?" General rules of either of 
these, kinds caji also be constructed if the student mentions an^ 
irrelevant factor or gives an incorrect value for a factor. 

Entrapment into a Prediction occurs when the te^chet asks for a 
prediction base^ on facttors ,that are likely to lead to an incorrect 
prediction. ' For example, ehet teacher might elicit a prediction 
based on insufficient facers that^ they grow rice it^Southern 
Florida because it is warm and \^ist (even tl^ugh it doesri^t produce 
rice). The teacher might elicit a\prediclSylon based on unnecessary 
factors that they do not gr^^ rice l>n Egypt because it is quite dry 
(even though it does produce rice). - Siih^larly inqorrect predictions 
of either type can be elicited for irrelevant factors or incorrect 
values of factors. . \ 



Entrapment based on factors OQOurs when\he teacher asfcs if 

\ ' ' ' 

jes of factors are consistent witlV^ or support a 

e of the dependent variable. For example, entrapment 



* Table 7 

Different Types of .Erttjrapm^nt Strategi 

Entrapment on Rules 

\ 

'Rule' based on insufficient factors 

\ .. . . 

Rule based on unnecessary factors 

Rule based on irrelevant factors - 

Rule based on incorrect values of factors 

« . ■' * 

/ ^ , 

Entrapment on Predictions 

Prediction based on insufficient factors 
Prediotior; based on unnecessary factors, 
Predictiorl based on irrelevant factors 
Prediction based on 'incorrect values of factors 

Entrapment on Factors 

Entrapment based on insufficient factors t 
Entrapment based on unnecessary factors 
Entrapment based on irrelevant factors 
Entrapment based on incorrect values of factots 



on insufficient factors might occur if a teacher asks whether the 
warm climate and' flat terrain in Florida accourfts for their failure 
tp J grow rice there. :^Entrapment on unnecessary factors 'might' occur 
if a teacher ask^ whether the lack of rainfall is consistent with 
their growi'.ng r ice , in ETgypt .. Similar kinds of entrapments can be 
constructed far irrelevant factOrS and incorrect rvalues of faetors. 



Insert Table 8 about .here 



Table 8 shows the condition-action pair for an entrapment based 
on irrelevant factors. .There are two conditions together that 
trigg-er the use of su9h a strategy: if a particular value of ^the 
dependent variable is being considered, and there are irrelevant 
factors that the student might consider relevant. The action ', taken 
'is to question whether. the irrelevant factors are consistent with or 
support the particular .valiie of the dependent variable being, 
considered. The example shown is from a medical^ dialogue given in 
Swets and Feurzeig (1965). s/ ' 

Identification and Evaluation Strategies . There is a large 

variety of strastegies for trying to get students to identify^- and 

evaluate different cases, factors, rules, and predictions. We have 

identified a dlmensi.onalized space of strategies teachers use for 

questioning students and commenting on their answers. Our proposed 

i 

space of strategies is shown in Table 9. 



4» 



Table 8 

• 

A Condition-Action Prair for an Entrapment Strategy 

' Entrapment based on Irrelevant factors ^ . 

- -3 — .• .0 ,. . , ^ . ' . 

If (1) a particular value of the dependent variable is being 
^ -^considered for a case, and ' 

(2) there ar§ one or moi^e irrelevant factors "that a student 
might consider relevant, . ' 
then (3) ask if the valtfes of the irrelevant factors are 
consistent with the yalue of the dependent var\a 



EXAMPLE (from Swets and Feurzeig" on medical diaghosis) 

T. Pleural pain, dyspnea, fever, and the physical, exam signs 
ar* certainly consistent, with pulmonary infarction. (Point 
out correct values of factors) Do you think that ^biking 
chills ^, and the presence of rusty gputum further supports 
this diagnosis? (Entrapment based on irrelejyuft factors) 
■ S. (Student resists entrapment) 

T. Right. „ . » 
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Insert Table 9, about here 
\ 



The two major dimensions in Table 9 are the type of strategy' 
(identification strategies vs,"^ evaluation , strategies) and the 
Q^jgg^ *^he strategy is applied to (either a dependent /variable, 
^ rule, fSsctor, or -case) . The table collapses three different, kinds 
of identification and evaluation strategies: 

1. Questioning strategies; Ask for or Ask if! ' * 

^, 2. Suggesting strategiesi^^uggest 

3. Conunenting strategies r Point out. , ' * 

The table is presented, in terms of the questioning strategies: . 
-i.e., ask for. or ask if. But each rule in Table 9 can occur in the 



two other forms obtained by replacing Ask by Su9;g^st or" Point out> 

The suggestion form of each rule"- occurs whenT^e teacher doe's not 

try to elicit the information from the student, but inst^d proposes 

a factor or a value of the dependent! var iable for /the student to 

consider, without telling the studentt whether the proposed 

information is correct or not. The commemsv^orm of each rule occurs 

when "the teacher simply tells the student what .the correct 

information is. jln inquiry dialogues the question foi;m of each rule 

is most common I but the other two forms do occur sometimes: the 

suggestion form when the teacher wants students to think 

h'ypothetically,. an<3 the comment form when the teacher doesn't' think 

" f 
the ^student can generate the information. 



. Table 9 



Identlf icdtion gtratectles 
Dependent Variables .; . 

Ask, |or . value of depenqlent. 
var iable : . 

Rules 



■ Ask fpr the formtslation of' 
^ a r u le . , 

Ask for the formulation of 
an alternative rule 

Factors - 



Ask foil^ suf f icient factors 



Ask for necessary factors 



Ask for relevant factors 



Ask for values of factors 



Eval uation Strategies 

" ■ • ■ » I*" 0 

■ ■.• , ♦ 

*Ask if a value of dependent 
. variable is' correct ot incorrect 

Ask if a rulie ia correct 
or incorrect 

t 

Ask if a rule is the same or 
.different from another rule 

♦ 

Ask if f acto'rs are 

sufficient or 'insufficient 
Ask if fetors are 

necessary or unnecessary 
AB^ if factors are 

relevant ot irrelevant 
Ask if the values offactorfe 

are correct or incorrect 



Ask for prior steps 

Ask for intermediate steps 



Ask for subsequent steps , 



Ask if a step is a prior step 
Ask if a step is an iijtermediate 
step 

Ask if a step is a subsequent step 



Ask for a-imilar l^tiea in 

factors for 4|l|h.lar ca!3es 
Ask for differences in 

factors fbr similar cases . 
Ask for similar»ities in 

favOtors for dissimilar cases 
Ask for dj^ferences in 

factors for diqisimilar cases 



1 



Cases 



Ask if, similar bases are 
the sarile on*given factors 

Ask if similar oases are 
different on given factors 

hsk if dissimilSr cases are 
« the same on given factors 

Ask if dissimilar' cases are 
different on give^' factors 



Asjc foi? a case witih a given 

value oh the dependent variable ^' 
As^ for a case with given 

values on some factors 
Ask for *a case with given f 

values on some factors^and 



on the dependent Variable 



In dialogues where thfe teachers are trying to encourage 
verbalization, such as thos^ of Warmari and Schank, twc* other forms 
of the identification cules occur: rewarding strategies and 
reformulating strategies. There is f|^equently ^ rewarding of the 
student when they formulate a rule, identify a factor or case, or 
make a correct prediction about the dependent variable. In Warman's 
case she rewards the stu<ients by telling them they have has a good 
idea, and then either repeating or reformulating what they said. 
Reformulation also occurs in situations where the student's' 
statement is underspecif ied ; e.g., the. valines of - factors are 
implicit rather than explicit. In one case, we have,' seen a ' teacher 
use a negative reward strategy, by pointing out that the student was 
failing to formulate a general rule. But much more commonly the 
teachers stayed with positive rewards and reformulations. 

* V 

within the object dimension, there are a number of 
subclassif icatipns we will describe. In identifying and evaluating 
different factors, there are three different subgroups. The first 
subgroup repeats the pattern found among the counterexample and 
entrapment strategies: i.e., there ar/^ questions abou t ^ suf f ic lent 
factjors, necessary factors, relevant factors and the values of 
factors. This pattern reflects the logical relations among 

different factors. 

» 

The second subgroup derives from the structure of logical 
chains (see Figure 1). Questions can bej posed to elicit steps prior 
to some node in the chain, steps int^jrmediate between two nodes, and 
steps subs^^u^^nt ^to some node in the chain. These strategies 

' * ■ ' . ■■ . 
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usuitlly o^^cur when the teacher" is pribing for a caysal chain linking 
together different factors, but can occur. when a teacher, such as- 
Socrates, traces a logical chain to the point wliere the student's see 
a contradiction betwfeenythe implications of'what they said and what 
they believe . . , • , . 

* * 

The third subgroup of strategies arises out of sUu^tl^ns where 
the teacher has selected two'cases and asks for a cbmparison of 
factors. These strategies correspond to the. case comparison 
strategies in Table 5. If the two cases have the same value on the' 
dependent variable, then asking far similar itiePforces the student 
to abstract the sufficient factors leading to the value of 'the 
dependent variable (i.e., generalization). Asking for differences 
in this situation .forces the student, to notice what factors do not 
change with the dependent variable (i.e., aje unnecessary). If the 
two cases dyfer on the dependent variable, as'king for differences 
forces the student to abstract the necQssgry factors that lead to 
changes in the dependent var^iable ( i . e . , di f f er en t ia t ion) . "Asking 
for similarities in this situation forces the student to notice what 
factors do not change given, a change in the dependent var iable 
(i.e., are insufficient).. Thus thes^ strategies allow the teacher 
to focus the student on different necessary and sufficient factors. 

.There are three differen^t strategies for asking the student' to 
identify given case. These derive from distinctions between, the 

dependent' var iables, the factors, and the rules f slatting factors to 
the dependent variable. Thus you can ask a student to identify 'a 
case with a .given value of the dependent variable, a case with given 
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valued 9f parHiicular factors, or a casQ with given values both for 
particull^ factors and the dependent variable. There .a^e no 
evaluation stragegies cor responding to the identification strategies 
for cases. This is because they are redundant with other evaluation 
strategies. For example, the evaluation strategy that "^would 

correspond to the first case identification strategy is the same as 

/■ . ■ . ' 

the evaluation strategy ^Ask if a value of the dependent variable .is 

correct or incorrect". The redundancy occurs because the strategies 

for dependent variables, rules and factors all assume a c^se has 

been specified. « V ' 

Insert Table 10 about here 



Tables 10 and'll show the condition-action pairs for two oif the 
•strategies in Table 9* The first shows an iden t if ica tion ' s tra tegy , 

"Ask for intermediate steps", This can , be eliteited whenever . a 

\ ■ 

student specifics two nop-adjacent steps in a causal chain. Iti the 

I. 

\ 

question form of the rule the teacher then asks for the intermediate 
ateps, but alternatively the^ teacher might suggest or point out the 
intermediate st§p. . The example shown is from Stevens and Collins 
(1977) where the tutor was teaching the factors that lead to heavy 
rainfall in Oregon and the student left out an intermediate step in 

* 4 

the tutor's causal model. * > 




\ 



Table 10 I 
A Condition-Action Pair for ag Identification Strategy 

Ask for intermediate steps 

$ ^ 

If (1) two steps in a causal chain or procedure that are not 



adjacent have been identified, • ; " ■ ■ 

then (2) ask the student to identify the intermediate steps. 

EXAMPLE (from Stevens and Collins on causes of rainfall) 



8. When the ji^oisture laden airj reaches the mountains it is 

, for'ced to rise and consequently the air cools? Causing 

rainfall', no? ' ^ 

T. Why does cooling cause rainfall? (Ask for intermediate 

steps) _ ^ tr. \ ^ 



Ins^t Table 11 here 



r 



Table 11 shows the - commenting version of the evaluation 
strategy, "Ask if a set of factors is suffici^t or insufficient". 
This variant can be triggered either by the student makii^^ r.^ 
prediction based on insufficient factors o/ by asserting that a set 
of factors is sufficient. in this variant the teacher points out 
that the set of factors is insufficient. The example shown is from 
Swets and Feurzeig (1965) where the student's task was to i'dentify a 
letter from a set of features. When the s.tudent made a prediction 
based on insufficient data, the tutor pointed out the insufficiency. 

Tables 5, 7 and 9 then summarize the set of strategies (155 in 
total Counting all the variants) that follow from tkc structure we 
have develoj^. But these do not cover what\ teachers dQ 
exhaustively. They cover about 80% to 90% of the teaser's 
statements in the dialogues ; we have analyzed. There are other 

i 

things that teachers do, such as getting students to test 
hypotheses, pr to questipn authority, that are not incorporated in 
this space of strategies. Nevertheless, this taxAnomy captures ^ 
very large part of what the teachers we have Studied are doing at 
the local level in order to carry on a dialogue with their students. 

M 

Dialogue Control Struc ture | 
The control structure that the teacher' uses to allocatiL / time 
between dj.fferent goals and subgoals may be the most crucial aspect 



Table 11^ 



A Conaition-Action Pair for an Efvaluation Strategy 



Point ^ut__fac to r_8 are inguf f icient 



If (1) a student makes a prediction about' the' value of the 
dependent variable 'based on a set of factors that are 
insufficient, or 

(2) asserts that a set of insufficient factors is 

f 

sufficient, ' 
then (3) point out that the set of factors is insufficient. 

EXAMPLE (from Swets and Feurzeig on identifying a letter) 

T. Start when ready. (The student niust guess a' letter from its 

> 

features.) «^ - 



S; Cur'ves? 
T. One, ' 
S. Loose er\ds? 

T. Two, ■ 

S. Obliques? ' j 

T. Zero. ' / 

S. c'.. , . ^ • • 

T. Ydu don't have enough information yet to get the right 
answer. (Point out a set of factors is insufficient) How do 
you know it isn''t J, fori example? ('Suggest a value of the 
dependent variable) ^ 



of effective teaching. An- earlier atteiflpt at a theory of the 
control 8bT,ucture was developed' in' Stevens and Collins (197^) . That 
theory -was based on protocols taken from teachers while they were 
teaching over linked computer terminals. The four basic parts of 
the control-structure theory are: (1) an agenda for keeping track 
of different goals ind subgoals, (2) a set of priority rules for 
adding goals and subgoals to the agenda, (3) a set of strategies for 
selecting c^aes- with ' respect to the high-level goals, and (4) the 
teacher^'s model of the student. 

The Agenda. ]\3 we have argued elsewhere (Collins, War nock and 
Passafiume, 1975b), the agenda that guides teachers in their 
selection of topics is ^not prespecif ied , except in terms of a few 
global goals. For the most part the agenda is constructed as the 
dialogue progresses on the basis of the responses of the student, 
the high-level goals of the teachers, and the previous topics 
discussed in the dialogue.' We will tfry to describe briefly how this 
complex interweaving occurs, though it is^ described in more detail 
in the earlier paper. • 

■' , ^ • • ^ • ' . 

The high-level goals of the teacher are the topics specified on 

, ' • V 

the agenda before any dialogue occurs. In Schank's dialogue, there 

appear' to be three top-lev.el goals which manifest themselves as 

phases of bhe dialogues: t^o define what a plan is, to specify the 

primitive types of possible plarrs in terms ot the definition, and. to ' 

analyze cases of planning in terms of the primitive types. These 

three goals in turn, derive from Schank's top-level goal of teaching 

v.. * ^' . . 

Students how to^ construct th<K)ries: they are the' subgoals 

■ , ' • . -36- ■ \ ' 
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instrumental to that goal, a similar age^a occurs in jfehe Anderson 
dialogue off factors affecting average temperature:^ the first phase 
of the dialogue was directed tioward getting the student to f6rm the 
hypothesis that distance-f rom-the-ocean affects temperature, and the 
.second phase related. to how the student could test the hypothesis. 
Similarly these two goals derive from Anderson's top-level goal to 
teach theory construction.. In m^any of the dialogues there is on]^y 
one phase' (as in the Warman dialogues or the Stevens and Collins 
dialogues), but there is always some high-level goal driving the 
dialogue. ^""^ ' , . 

These high-lfevel goals guide the select ion s of cases (see 
section bel<fvi on -global strategies for case selection) and the 
generation of 'specific questio.ns to probe fbr predictions, 'factors, 
and rules (usually in that rorder) about, thie cases- selected. 
Students'- resp9nses to these questions ip turn spawn local" subgoals 
to diagnose and correct the misconceptions and omissions revealed by^ 
^hem (Stevens ^nd C^pllins, 1977). These subgoals are then added to\ 
the agenda according -to a set of priorities given inthe'next 
section. . . ^ 

t 

can illustrate the way the agenda works most clearlV with- 
reference to a dialogue given in the Ste.vens and Collins paper. The 
dialogue was ojn the subject of what factors lead 'to rainfall in . 
different places, and the case selected was a paradigm case of heavy 
rainfall, namely the Oregon coast. The teacher first asked for the 
student's prediction, and went on to adk about ^the causal factoi;,sv 
leadipg to heavy rainfall in Oregon. I In one response the student ' 
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made a, guess based oh what she had learned about the Amazon, "Pees 
the air (n\pi^t air) from the ocean somehow get blown over Oregon and 
encounter a block o^ some sort which causes it to rise and cool?". 

r J 

The. teacher commented off-line that the student's answer "was missing 
^three basic st^ps in the^ teacher's first-order theory of the factors 
• leading to rainfall: (1) why the ^air is moist, (2) why it is blown 

over Oregon, and (3) why cooling results in rain. Then the teacher^ 

J 

( 

asked about the first of these steps "Why is the air so moist?" , and 
^ held the other two on his agenda. The di^cussiort .of this topic 
continued for interchanged, with additional subtopics added to 
the agenda during the discussion, before^e teacher returned to , the 
second* ^miss/ng step (i.e., why the air is blown over Oregon) 'oji the 
agenda. The third step was raised sholtly thereafter. 

. Our analyses indicate that the agenda is ^ ail order^ list -of 
, , goals ^ which' are held until they aj:e satisfied. When a gpal is 
sati^sfied, it is removed from the agenda and the next goal is 
pursued. New go^ls can- be inserted, at arbitrary places in the 

* • 

agenda and it can be reordered. However^ we expect that 

{■ 

•* 

•manipulations other, than pushing new goals on to. the beginning and 
» popping . them off of the beginning require extra efforts on the. part 
of the teacher and therefore the tendency is to treat the agenda as 
a push-down , stack pushing and popping goals off of the top. 

This'pattetn of pushing and popping of different ''goals is 
evident in almost. all conversation (Collins, et al., 1975b; Grosz, 
1977; Levin , and Moore, 1977? Reichmaa, 1978)-. When pop-upfe occur 

i 

they are often signalled by , various clue words such as *'Okay"r 

ERJC' • • ' 5,', ■ 
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"Now", or-Anyway"^ (Collins. et al., 1975brReichman, 1978)'. Ther^ 

is some suggestive evidence tWat well-prepared teachers ' come in, with 

a highly-structured theory of the domain they are teadhlng, At\6 

'frequently select new tbpics from the ,ag*e>nda to cover different 

aspects of the theory, rather ' than following, topics assqciatively 

from one to the otheS, as do less well-prepared teachers (Collins et 

'• . ' ' • 

al.., 1975b) . ' 

Priorities for adding goals to the agenda i Tn adding goals to 
the agenda,** there must 'be a set of priorities. Often a single 
question, as in tl^e example above, uncovers several misconceptions 
or omissions that the teacher may 'want to pursue." In such 
circ.umsta,nces, the teacher must, decide' which to pursue first. But 

♦ 

eveo when goals are added to the agenda at different times, it- is 
essential to decide which should pursued first; that is to say 
the teacher may want to pursue a new goal being added befoje some 
other gdal already pn the agenda. Or the teacher may want to drop 
some previoas . goal, irf favor of some ° more impo\tant goal* he 
identifies during the dialogue. For alf these reasons,] there must 
^be a set of priorities for order ing- goals a)a the agenda 

In Stevens and Collins (1-977) we identified " four priorities 
that occurred among the teachers'' comments on why they were asking 
each q.uestion. in the dialogues with grpups^ of students. , a rifth: 
priority for 'allocating t/me ' between >aiffereht stud^^^ 
■apparent. We 'list the pr i^^ i ties _ belW/rovjg-hly^ 
highest priority first, but we th'ink 'peyN^.A^re-,' ^ft^i^hed [tog^hey^A' in- 
makinq a decision about wh^ch ■g^j'aU t^^Vptir f" i^^/st. ' . v 
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Errors before* omlaq^lona . Teachers correct any errors 

they ^-i^cfncpse before they deal with omissions in their 

theory. This is because errors have more deovas^tat ing 

consequences; they can interfere with learning other 

information correctly. This pr-iority ij3 sometimes 

violated when the teacher is trying to l^ad the student 

to discover his or her own ercor by an extended 

■>.■■* ' ' ' ■ ' ' 

dialogue, during which moi^.e immediate goals arise. 

" . ' ■ • ■ . ■ 

Shorter fixes before longer fixe s. Teachers typii:ally 

will correct errors^or omissions that can be taken care 

of quickly, before taking on more extensive problems. 

For example, teachers will often deal , with errors 

.about facts .'or about principles oOtsj^de the domain 

being taught, by simply telling the student the correct 

answer (Stevens ,and Collins, 1977.) . This is done 

% . ■ " - 

quickly, so it tends *'to take highest priority. But 
shorter fixes do not always take precedence; for 
^J^^fple ' in . the* dialdi^ue on rain in Oregon^ the second 
and third steps were dealt with, much more quickly than 
the first- step, which took precedence^ . because of 
priority 3. 

■ • * f 

Prior steps before later steps . Teachers. of ten take-up 
s^teps- in causal , chains in* the order of occurrence/ so 
that the fliscupsion moves in the ordetr of temporal or 
causal sequence. Ws^s what the teacher did' in the 
Oregon example when he identified three missing steps 



in hie causal '^thepry, and t;hen asked abouU the firsf 

nlissing step. This may often, be .violated where 

..branching structures occur in the theory, -'*pr the 

student brings up a ► causal factor somewhere in the 

middle pc at the end of the chain. 

I 

Low-order factors before high-order factors . Teachers 
select more important (i.e., low-orde^r) information' 
before less important' (i.e., high-order) information, 
particularly when a pop-^up occurs and they have to 
select a new ^rancK of a causal structure to pursue. 
This priqrity is violated frequently in that teachers 
pursue branches or" subparts of a causal model in detail 
before taking up other branches. Tl^is occurred in the 
Oregon dialogue where the teacher went into a fair 
amount of detail about currents and evaporatioij in 
pursuing the first missinq step, bajbre taking up the 
second and third missing steps. 

✓ 

Students who haven^t spt)ken for awhile before students 
who have . This is most noticeable in the Schank and 
Beberman dialogues where the teacher is trying to get 
all the students to participate and verbalize their 
theories. It is violated when the^ teacher is pursuing 
a particular lipe cf • inquiry with one oi another 
student. , 



✓ /. 

These five principles are the ones we have heeWable to 
identify in the dialogues, but there may be othef princlttles that 
are also contributirag to the order in which goals^are adde^ to the 
agenda. 




Case selectio^n with respect , to high-level go4i3> <GlW a set 
of high-level goals, the teacher selects cases that opWm^e the 
ability of the student to master those goals. T,here appear to be 
several , overall strategies that teachers apply in selecting cases: 

1. Select cases that illustrate lower-otder factors before 
higher-or„der factors. For example, in teaching about ' 
rainfall, Collins and Stevens move from cases l-ike the 
Amazon and Ireland that exemplify a first-order theory 

to cases li'ke Eastern America or Patagonia where the 
factors .are more complex. in teaching the distributive 
law, Anderson chose cases that systematically isolated 
, one factor after another. 

2. Select more salient or more frequent cases before less 
salient or less frequent cases. Other things .being 
equal, a South Amefican geography teacher will select, 
cases like Brazil and Argentina rather than Paraguay 
and puyana. A medical professor will select the most 
frequent ^diseases and the ones that are most important 
to diagnose. 

3. Select cases such that a generalization will make ' 
"prediction easier or les,s tedious. This Vs most 
evident in the Beberman and Anderson transcripts where 
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they prfssenfe a series of math problems that are rather 
tedious to work unti 1 .you see the short cut (see Neches 
and" Hayes, 1978 for a discussion of -strategy 
modification) that leads to a significant 
generalization, in one case the rules for addition of 

T ' 

I 

real numbers, in .the other the^ dis^r ibut ive law* This 
case selection^ strategy has its analog in 
non-procedural domains when the teacher selects a set 

i 

of cases that have some gener al ization that makes 
prediction easier. Fpr example, in the domain of 
factors affecting rainfall the teacher might select a 
'Set of cases like Southern California, Northern Africa, 
Northerrf Chile-, Western Australia, and Namibia in or(i^r 
to lead the student to induce the generalization that 
any place on the western. side of a continent in the 
latitude of 20 to 30 degrees will have little rainfall.. 
Select well-known cases that arise in the student^s 
experience. This strategy is most evident ,in the 
Warman dialogues wi^h preschoolers, where she selects 
cases that arise in the coiirse of school (concerning , a 
problem about playing with blocks or a movie they saw) 
to get the children to generalize about moral actions. 
This same strategy is^ apparent in the selection df 
well-known caSes by Schank, Anderson, and Collins* 
Because students have more knowledge in tljjjpse cases, 
they are better able to consider all the relevant 
factors and to abstract rules relating the factors to 
the dependent variable. 

-43- 



I The student model . The teacher's model .of the individual 
student guides the selection of y^hich parts of the domain to 

'discuss, which parts to 'skip over besause they will be too difficult 
for the student to assimilate, and which parts to assume the student 
knows (Collins, et al., 1975). it also guides the teacher in 
attempting to diagnose the stud6nt'sj)^isconcept ions about the domain 
(Stevens, Collins, and Goldin, 1979; Stevens and Collins, in press). 



We assume that the teacher has two types of a priori 
information that are used in constructing models of individual 
students: (1) a structured theory of the domain and vattached to 
each element in the theory (i.e., each node or link) the relative 
likelihood that any student will know alpout thjat element," and (2) a 
set of underlying misconceptions (alternative rules or theories) 
that different students -might have. We will discuss , in turn how 
these two kinds of .information guide the selection of goals to 
pursue. 



.When we saytth^at the teacher has a notion of the relative 
.likelihood that any - student will know a given element, we do not 
mean that the teacher thinks there's a 30% chance student A will 
know one element and^a 20% chance another element. Rather we assume 
only a partial ordering on the ^elements, reflecting perhaps when the 
teacher learned each element. This partial ordering corresponds to 
the notion of first-order to nth-ordeo factors in a theory. For 
example, the factors affecting temperature of a place might be 
partially ordered as follows: latitude,- altitude, ocean currents, 
distance frqin' the ocean, cloud and tree cover. The teacher's 

4 
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.aaaumption is that students l^arn the elements in at)proximately this 
•same order, therefore it is possible to gauge, wbat thp student will 
know or not know based on a few correct and incorrect responses. 
These responses are used to determine the criterion point in the 
t)artial ordering above which the student is likely ,to know any 
element and below which the student is unlikely to know any element. 

In our earlier work (Colling, et al., 1975b) we noted four 
levels of elements with respect to this criterion point in the 
partial ordering that determine the goals the teacher will pursue. 
We ijepeat' those here in terms of our Current framework: 

1. Eleme.nts the teacher can assume ?he student knows, an^ 
henqe need not j^ursue. 

2. ^ Elemjmts -the student may know, and so the teacher asks 

the st^aie4T^_Jfe4 provide them (i.e., give predictions, 
factors, etc.) These include all the elements just 
above and below the criterion point. 

3. Elements the student will not be able to figure out., 
and so t>ie teacher will tell the student if they come 
up in the dialogue. 

4. >')Elements so far beyond the student's current level, that 
they can not be assimilated until the student has more 
ijjformation. These elements are not mentioned by the 

teacher . ^^^^^ 

Thus these Levels determine what goals will be added to the agenda 
Ji.e., goals will be added for teaching elements at levels 2 and 3) 

^ ^0 ■ y 



and whether these'wHl be pursued with questions (level 2) or 

oomments (level 3) . ' ' 

.. • ■ • . 

The experienced teacher also accumulates a large amOunt of 
know-ledge about the possible misconceptions students may have. In 
the domain of rainfall,*^ we (Stevens, et al.< 1979) have identified 
^ sixteen basic misconceptions students have, based on systematic 
questioning of eigtvt high school students. « In arithmetic Brown and 
Burton (1978) have identified the 50 most common procedural errors 
that primary students^ have, based on data from 1300 Nicaraguan 
children. Depending on the question or the problem, a particular 
misconception can appear in many different forms, so that a teacher 
must recognize a variety of manifestations for each possible 
misconception. 

We think that teacherp store many of the misconceptions they 
gee as perturbations of 'subparts ^ the knowledge structure they are 
trying to teach. When a student makes a misstatement, the teacher 
may recognize it as a m.^ni testation- of one of the possible 
misconceptions. If the teacher can not identify the misconception 
underlying the error, or if there is a pattern of misconceptions 
• thra^t frequently occur together (Stevens et al, 1979) , the teacher 
will asK questions to ident tfy -w^hat t-he underlying misconceptions 
. are^ In. order to .correct any misconceptions there are a variety of 
actions the teacher' m^y take (Stevens and Collins, 1977) . The 
teacher may simply inform the student of the correct answer, or ' if 
he teacher thinks the student won't get further confused. 




counterexamples, hypothetical cases, or trailing consequences may. be 



U3e.d to Gncp'Urage the student to debug his or her own mlsconception.v. 

CONCLUSION / 

This summarizes what we think are the most important elements 
■•»• » 
.of effective teaching. By turning teaching into problem-solving in 

this way, by selecting cases that optimize the abilities the teacher 

d 

is trying to teach, by making students grappl/5 with counterej^mples 
and entrapments, the students are challenged more than by any , other 
teaching method. Because of the experience they are able to attack 
novfel problems by applying these strategies thapiselves. 



f 



Appendix ° 
TEACHING STRATEGIES IN THE THEORY 

» ' ■ ■ 

9 ■ 

^ Case Selection Strategies 
CSSl: Positive paradigm exemplar for factor^s 

N . ■ ■ 

If (1) a student has not identified many of the factors that are 

relevant to a particular value on the dependent variable, ' 

then (2) pick a case where as many as possible of the value?> on the 

factors are consistent with the particular value on the 
dependent variable. ' " 

EXAMPLE 

If a student is being. taught the factors affecting whether a 

place has heavy rainfall or not, pick a case like the Amazon or 

Oregon where all the factors have values that lead • to heavy 
■ rainfall. . \ 

CSS 2: Negative paradigm exemplar for factors ' 

If (1) a student has not identified many of the factors that^' are 
relevant to a particular value on. the dependent variable, 
then (2) pick a case where as many as possible of the values on the 
factors are inconsistent with the particular value on the 
, dependent \?&riable. \, 
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EXAMPLE 



If a student is being taught the factors affecfting whethef a * 
place has heavy rainfall or not, p^ck a case like the Sahara or 
Southern California where all the factors have values that lead 
to little rainfall. . ' ' , 

/ 4 

CSS3: Positive exemplar for a sufficient factor (Near hit) 

If (1) a student has not identified a factor that is sufficient 

for a particular value on the dependent variable, 

then (2) pick a case where the factor is predominant, the value of 

the factor is consistent with ti^ given value of the dependent 

variable, the values of the " «ther sufficient factory are 

/ ■ J) <, ' 

inconsistent with the given value of the dependent variable, 

'"X ■ ' * 

\and the dependent variable has the giveA value. 

EXAMPLE 

SuppoW a teacher wants a stud^n| to see that you don'^t need 

rainfall for growing • r ice . Then the teacher might choose Egypt 

which * has little rainfall, but does grow rice by using\ 

irrigation from the Nile. ^. 

CSS4! Negative exemplar for a necessary factor (Near miss) 

If (1) a student has not identified a fabtor that is necessary for 
a particular va\ue on the dependent variable^ . 

then (2) pick a case where the factor is predominant^ the value of 

the factor jls inconsistent with the given value of the 
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dependent yariable, the values of the other factors are 
consistent with the given va.lu6 of the dependent variable, / - 
and the dependent variable has the opposite value. 

EXAMPLE (from Collins on factors affecting population densit^^) 

T. (In discutrsing population density, tfe student had' not 
identified climate as a factor.) OK. Now do you think it's 
very .dense in Alaskk? (CSS4: Pick a negative exemplar for a 
necessary factor) 

S. No . ., ' , . • . 

I ' ' . 

T. Why? (IS6-^/ Ask for relevant f-actors.) 

S. I would imagine because of the cold? % 

^^^5: Generalization Exemplar for factors (Maximal pair) 

If (1) a student has not identified one or more factors that /are 

I V f 

•relevant to a particular value on the dependent variable, ^nd 

(2) there 'is a case identified that is a positive or -n^ative 
exemplar for those factors, ^.^^ - ' 
then (3) pick a case that has the same or similar values as the 
previous case on the given factors, that has as. different a 
value as possible on other factors, and that has the same or a 

simijiar value on the dependent variable. 

-** • 

EXAMPLE -(from Stevens and Gollina on causes of rainfall) 
< / * 

! ' ' • / 

T. The' current is called the Japanese current and it comes from 
the; Equator along the coast of Japan/and across to Canada and 
Oregon. (IS8bj Point out prior^^teps) Is ^here another 



i 
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current you know about with the same patte/n? (CSS5: dick a 
generalization exemplar for a set of factors) (ES14: Ask \or a 
case with^iven values on a set of factors) ' • * 
- S. I don't know what you mean - t:he equatorial current? 

I meant th% Gulf stream. (IS16b: Point out a case with given 
. values on a 'set of factors) I wanted you to see the general 
\ pattern of currents inVhe world. (ISllb: Point \ut 
similarity in factors between si|iilar^ cases) . 

Differentiation exemplar fojr'gactors (Minimal pair) 

If (1) a student has not identified one or more factors that are 
relevant to a particular value on the dependent variables, and 
(2) ' there is a case identified that ih a po\itive or |egative 
exemplar fdr those factors, 
then (3) pick a ca^e that has a different value frol^ the previous 
case on the given factors, that has the same or simiolairj valines 
on other Actors, and that has a different value 
dependent variable. ' 

EXAMPLE (from Collins on the factors affecting population density) 
T. 0^. Why do you suppose Java has a high population density and 
some of- the^ other Indonesian islands have low population 
density? (IS14: Ask foe differences in factors between 
diffeci^nt cases) 
S. There's so many of them. ^ 

; 

T.Sumatra, (CSS6,: Pick a, differentiation exemplar for / factors)' 
(Sumatra is chosen because it's like Java in most respects, 



r 

bn the 



C e.g., climate, location, but had a different ^value on the 
dependent variable.. This force? the student to pay attention, 
to the- factors, such as terrain, that differentiate jTava and 
" Sumatr a) . . - ' ■ 

CSS7: Exemplar to show variability of a factor " /\ 



If (1) a student has identified a factor that is N^ievant to\ a 
' particular value of the dependent variable, and \ 



(2) there is a case identified that halfe a*particular value on 
that factor, . ^ 

\ 

then (3) pick a cas^ that has the same value on , the dependent 
variable, that has as different a value as possible on the 
particular factor, and that has as similar values ad possible 
on the other factors. j 

EXAMPLE 

Suppose Jav$ has been identified as a place that is warm endugh 
to grow rice, then pick. a case lik0 Japan which is much cooler 
but still grows Vice. ) ' 

* 

CSS8: Exemplar to show variability of the dependent yariab le 

\ ■ 

If (1) a student has identified one or more factors that are 

* f 

relevant to a particular value , of the dependent variable, and 
(2^ there is a case identified that has a particular value on 
the dependent variable, 
then (3) pick a case that has the same' values on the factors, and 
that has as different /a value as possible orf the dependent 
variable. 
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Supp^ tlie^ CoKgo ; jungle .Wg beeTiUdeiikti&^i a-^ >^ piafee n^at^^'^^' f^^ 

.... • ■ ' ■ ■-^•;->.'V^ 

the .^equ^atbr ,wh&t;e: the '^v^ei age tempera tti^e as &S 'degV^esi ^d- :90 -^^^ 
degrees, -^hen piaH.;^^case^Xiloe;:;the ^-^op^Xiof/Vte;::^ KilXm^^ 
.which is' al$o near the egua tor but a 
much CO lde.r (<3 2 degree's) . . * • : . ' 



CSS9: Counterexample for insuff icient factors 



, If (1) a student proposes ^ 'ru le"W makes 'a pre^dictlon ba^eji "on 
one or mor^ fafctors that are insufficient, 6r " ^ 

(2) is en^trapped by a V^Kle (ENS 1 or ENS 9) based pn one or 
more factors that are insufficient, 
then (3) pl^k a case that has the values specified on the ' 
insufficient factors, but not the value specified on tie ' 
dependent variable. ^ - . 

EXAMPLE (from Collins on factors affecting grain growing) . 

T. Why? (i.e. why do they grow rice in Louisiana)' (IS5: Ask for 

relevant factors) • . . 

S. Places where there is a lot of water. I think-, rice requires 

the ability to selectively flood ' fields^ 
T. OK. Do you think there's a lot of rice in say Washington and 

\ 

Oregon? (CS&9: Pick a counterexample for an Insufficient 
Jtor) (ISl: Ask for the value c^f'the dependent variable) (T 
sets, a case where there is a lot of water buf no rice; this 
coi/nte^rexample then led the' student to consider climate and 
terrain) . > «• , 
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CSSIO: Counterexample. for unnecessary .factor s ■ 

■ •- * ■' - ■ ' ■> *•• 

.If (1) a student proposes a rule or makes a predicrti^on based on 

6ne or morje factors that are unnecessary, or . ' 

• (2) is enfc-r^ppec? by a rule (ENS 2 or ENS 10) based on one o^ 

more factors that are unnecessary, ; < 

.... / ■ 

then (3) pick a case that does not have the values specified on the 

unnecessary factors, but do'es have the value specified on the 
i' dependent variable. * - * 

• . ' - ■ 'J : 

EXAMPLE ' (from Collins on factors affecting grain growing) 

S. (In response to why they pan not ' ^xow. r ice in Oragon) I don"'t 
think the .land is flat 'enough. You've got, bo have flat land so 
you can fl'ood a lot of 'it. ' ' ' 

• , ■ ■ 

T. What about Japan? (CSSIO: Pick a counterexample 'foe an 
unnecessary factor) (ISl: Ask for the value of the dependent 
variable) (JafJLh- grows rice but does not • have . much flat, land.) 

CSSll:- Counterexample for' an irrelevant factor 

If (1) a student proposes a rule or makes a- prediction based on 
ohe or more factors that are irrelevant, or ^ " 
(2) is entrapped b^ a rule (e!nS 3 or ENS 11) based on one or 

^ ■ * ■ • 

. mor^ factors that are irrel^vant^ ^ 
then (3>-p*sJ(' a case that "has the values specified pn the irrelevant 
factors, but does not have, the Value sp.ecif ied ah the dependent 
variable, or ' , ■ 



\ 
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(4) picSk a case that does, not; have the values specified on. the 
irrelevant factors bat daes have the value specif ied on the 
dependent variable. 

. EXAMPLE 

Supposed student proposed that having * high humidity 'was 
necessary for growing rice or predicts that Java grows rice 
because of the high humidity, then the teacher can ask about 
Egypt where the humidity is low. but rice is t(rown, or the Congo 
wher^ humidity is high but^ no rice is> grown. 

# 

CSS 12: Counterexample for an ^ incorrec t value on a f^^ctor 

^ *^ ' " ' [ ' 

If (1) a student proposes a rule or makes a prediction based on 
^ • • one or more values of factors that are incorrect, or 

. (2) a student is entrapped by a rule (ENS 4 or ENS 12) based on 

\ . 

One or more values of factors th^t are incorrect, 
teheh (3) pick a case that has the values specified on the factors, 

V 

but does not' have the value Specified on the dependent 
var iable, , otC"''^ 

(4) pick a case that does not have the values specified on the 
factors, but does have the value specified on the dependent 
variable. , ' . " 

EXAMPLE V • 

* 

« » ■ 

Suppose a student proposed that having a" cool temperature is 
necessary for growing rice or predicts that Japan grows rice 
• because it is cool, the>i the- taacl^r.can ask about Java where 

. , y.. ' ' - ,, -55- • . 
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'the temperature is .quite warm all yea^ .around and they gr^w 
rice, or about Oregon which is cool but where notice is grown. 

CSS 13: Construcjt a hypothetical case for insufficient factors 

o 

•If (1) a student proposes a rule or makes a prediction based Qn 

^ , ' .... ^ • ) 

one or more factors that are insufficient, or 

(2) is entrapped by a rule (ENSl or ENS9) based on one or more 
factors that are insufficient,, "*■ 
then construct a case that has the values specified on the 

insufficient factors,, but not the valjues specified on the 
t dependent variable. ' 

EXAMPLE 4 

Suppose a student suggests they don't grow rice irr^British 
Columbia because it is too mountainous, ask the student '*If 
British Columbia were flat could' they grow rice then?". The 
answer is that they could not, because' of the cold temperature. 

0 

CSS 14: Construct a hypothetical case for unneces*^sarY f actors 

If (1) a student proposes a tule or makes a prediction based on 
one or more factors that are unnecessary, or ^ 
(2) is entrapped by a rule (ENS2 or ENSIO) based on one or more 
factors that are unnecessary, 
then (3) construct a case that does not have the values specified on 
,the unnecessary factors, but does have the value spiedified on 
- the dependent variable. 



EXAMPLE- . f 

.Suppose a student suggests they grow rice in Louisiana because 
it. rains a lot there, then the teacher might ask "If it didn't 
rain a lot in Louisiana, could <they still grow rice there?". 
The answer is they co.uld by irrigating the rice paddies from 
the Mississippi River. 

t 

GSS15: Construct, a hypothetical case for irrelevant factors 

* If iX) *a student proposes a rule or malces a pr«?i(iction based on 
one or more factors that are irrelevant, or 

(2} i^ entrapped by a rule (ENS3 or ENSll). based on one or more 
factors that are irrelevant, 
then (3) conrsttuct^a case that has the values specified or\ the 
irrelevant factors, but does not have the vd'lue specified on 
the dependent variable, or 

(4) construct a case thjat dges not have the values specified on 
the i?Velevant factors, but does have the value specified on 
the dependent variable. 

EXAMPLE 

. / 

Suppose a child asserts that John's tripping of Sam was bad 
Ijfecause SanTj^roke his leg, then the teacher might ask whether 
John was bad even if Sam didn't hurt himself at all, or even if 

I , 

9 « 

Sarti had accidentally tripped over John and broke his leg. 
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CSS16: Construct a hypothetical case for Incorrect values' of factors 

If (1) a student proposes a .rule or makes a prediction )Dased on 
one or more values of factors that a4;;e incorrect, or 
(2)^ a student is entrapped by a rule (ENS4 or ENS12) based on 
^one or more .values of factors that are incorrect, * 
then (3) construct a case that has the values- specified on the 
factors, but does not have the valye specified, on the dependent 
"variable, or ^ 

(4) construct a case that does not have the values specified on 
the factors, but does have the value specified on .th« dependent 
variable. 



EXAMPLE (from Warman on who can play with blocks) 

t 

S. How about n^gir'ls play wib+i anything and boys play with 
everything. (This is one boy's proposal for a fair rule.) 

T. Ok. Let's take a vote. Boys, how about if you don't play with 
any toys here in -school? (CSS16: Construct a hypothetical 
ca^e for an incorrect value On a factor) (ES2: Ask if rule is. 
correct or incorrect) 
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Entrapment Strategies 

* 

ENSl: Rule ba^ed on ihau^ficient factora . • 

n . 

It (1) a student explains the value of the dependent" variable 
based on one or 'ir)o<e factors that are not sufficient, or 
(2) makes a prediction baaed on one or irto,re factors that are 
, not sufficient, 

then (3) ask. if it is a general rule that the "^dependent Variable 
must have the' value specified . given the values of the 
^nsufficient factors. 

EXAMPLE (from Anderson on factors affecting temperature)' 

S; (In response to a question about why he predicted Newf oundlan<J 
was colder in winter "than Montana) Newfoundland is further 
north, • 

T. Yes, Newfoundland is further north than Montana. (ES6b: Point 
out correct value of a factor) Are you arguing then, that if 
you take any two places in the Northern Hemisphere, the one 
which is further nor-th will have the lower average winter 
temperature? .(ENSl: Entrapment rule based on an insufficient 
factor) 

■ •■/ r ^ ■ . • • • 

ENS2: Rule basfed on unnecessary factors 

> ^It (1) a student explains the value of the v dependent variable 
based on oneXor more factors that are not necessary, or 
(2) vmakes a .prediction qased on one or more factors that are 
not necessary', ^ 
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then (3) ask if it is a general rule that the unnecessary factors 
Hist have the values specified given 'ihe value of the dependent 
variable. 

EXAMPLE 

_ . ■ * 

Suppose -a student says lots of rainfall is ^ reason for growing 
rice, or predicts that a place with heavy .rainfall grows rice, 
then ask "Do you think it is necessary to have heavy rainfall 
to grow rice?" 

ENS3: Rule based on irrelevant factors 

If (1) a student explains the value of the dependent variable 
based on one or more fac(o^ that are irrelevant, or 
(2) makes a prediction based on one op more factors that are 
irrelevant, 

then (3) ask if it is a general rule that the dependent variable 
must have the value specified given the value-s of the 
irrelevant factors. ^ 

EXAMPLE ' ' , 

Suppose a student says they grow f ice in China because of th^r 
oriental nature, or' predicts they grow rice in Mongolia because 
of their or iental ^nature, ask if it is general rule' that people 
with an oriental nature grow rice. ' 
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ENS4: Rule baaed on incorrect values of factors 

" ^ 7: . : f 

If (1) a student explains the value Vf the dependent variable 
based on one or more incorrect values of factors, or 
(2) makes a prediction based on one or more incorrect values of 
factors, 

then (3) ask if it is a general rule that the dependent variable 
must have the va.lue specified given the incorrect values of the 
fact'ors. _ 



EXAMPLE 



J 



Suppose a student feuggjsts that a place grows r io6\because it 
. has a dry climate, ask if generally a place must havV a dry 
climate to grow rice. /~ 

ENS5; Prediction baa^d on insufficient fac^tors 

• 1 — 

If.: (1) a case is selected, where- the valu^ oi the dependent 
variable is inconsistent with the-value of one or more factors 
that are not suf f icien t , ' and 

(2) the value of the (!(ependent variable has not beeh specillied, 
then (3) ask if the dependent variable has the value tha't is 
consistent with the values of the insufficient factors, or # 
(4) ask the student to make a prediction based on the 
insufficient factors. 

/ 



\ 



Vample (from Collins on .faJtots affecting average temperature) 

T. IS it very hot along the coast here? (points to PeruvVan coast 
near the equator, where jfhe effect of latitude is overridden by 
ocean currents.) (ENS5: Entrapment into prediction based on- 
insufficient factors) 
>. S. I don't remember ^ . ., 

T. No. It turns out there's a very cold current coming up the 

it bumps against FeruV and tends to make the coastal^ 

^ area cooler, although it's near the equator. (IS7b:. Point out 
values of factors) (isib: Point out value of the dependent- 
variable) 

Prediction based on unnecessary factors • 

If (1) a case is selected where the value' of the dependent 
variable is inconsistent with a value of one or moire factors 
that are not necessary, and 

(2) the value of the dependent variable has not been specified, 

then\^(3) ask if the dependent variable has thel value that is" 

Consistent with the values of the unnecessary factors^ or 
I - » , . 

1(4) ask the student to i^ake a prediction based on the necessary 

factors , 



EXAMPLEv 



Suppose Egypt has been selected to discuss r iceNgrowing, then 
the teacher can ask if the student thinks they^ can not grow 
rice there given there is little rain, or whether the student 
thinks they could grow rice or not. . 




BNS7: Prediction Ipased irrelevant factor 



s 



If (1) a case is selected where the value of the dependent 
variable is inconsistent with what the student would predict, 
given the values of one or more irrelevant factors, and 
(2) the value of the dependent variable has not been specified, 

then (3) ask if the dependent variable has the value that the 
student thinks is consistent with the values of the irrelevant 
factors, or ' » 

(4)) ask the student to make a prediction based on the 
irrelevant factors. 

EXAMPLE • . ji 

. Suppose a' student thinks an Oriental nature is necessary for 
growing rice, then ask "Do they grow rice in Mongolia, s^ince 
they have an Oriental nature?" or "Do you think they grow rice 
or not in Mongolia?" 

ENS8: Prediction based on incorrect values of- factors 

If (1) a case is selected where ^ the value of the dependent 
variable is' inconsistent with what the student would predict 

♦ ,. given the values of one or more 'factors for which the student's 
4 rule is incorrect, and * 

(2) the value of the dependent variable has not been spe/ified, 

then (3) ask if the dependent variable has the value that the 
student thinks is consistent with the values of the factors, or 
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(.4) ask the student to»make a prediction based on "the incorrect 

* • . f 

value of the factor. 

* i 

EXAMPLE ' . 

i • • _ ^ 

Suppose a student thinks a dry climate is necessary, for growing 
rice, then ask if they grow rice in Arizona since it has a dry 

t 

climate, or ask whether -they can grow rice in Arizona. 
ENS9: Entrapment based on insufficient factors 

If (1) a particular value of the dependent variable is being 
considered for a case, and 

(2) there are one or more insufficient factors that have values 
in^jonsistent with that value of the dependent variably, 
then ^(3) ask if the value^^^f the- insufficient factors are 
consistent with that value of the dependent variable. 

EWIPLE , , « 

Suppose a ptudent is considering whether they grow rice in 

Florida, ask if the varm climate would account foi% the • ' 
inability to grow rice there. '/ 

ENSIO: E n trapment ' based on unnecessary f actors 

If (1) a particular value of the dependent variable is being 

i 

considered for a case, and ' • 

(2) tihere are one or more unnecessary, factors .that have values 
inconsistent .with that value of the def>endent variable, 
then (3) ask if the values of tBh% unnecessary factors are consistent 
with the value of the dependent variable. 
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EXAMPLE ✓ 

s * * 

Suppose a student is ^considering whether they grow rice in 
Egypt, ask if the lack of rainfall would make him think they 
grow rice there. 

ENSll: Entrapment base/on irrelevant factors 

If (1) a particular value of the dependenit %, var iable is being 
considered for a case, and 

(2) there are one or more irrelevant'' factor s that a student 
might consider relevant, • • 

then (sj.-ask if the values of the irrelevant factors -are consistent 
with, that value of the dependent variable. 

EXAMPLE (from Swets and Feursjeig on medical diagnosis) 

T. Pleural pain, dyspnea , • fever , and the physical exam signs are 
certainly consistent with pulmonary infarction. (ES7b: Point 
. out values of factors are correct) Do you think that shaking' 
chills' and the presence of rusty sputum f^irthfer supports this 
^diagnosis? ' (ENSll; Entrapment based 6n irrelevant factors) 
S. No. • ' • * A 

T. Right. 

J|NS12: Entrapment based on incorrect values of factors , ' ' 



> <r 



If (1) a particular value of the dependent** variable is. bfeing,' 

considered for a case, arid • / i'' 'f 

, ■■' ■■■■ > ■■■ 

(2) there are values of one or- more factors that are 
7* ./ ^ , , - 

inconsistent with that value of the dependent .\^ariable/ 



then (3) aak if the values of the factbrs are consistent with the 
value of the dependent \far iabfe'. 



EXAMPLE 



Suppose a student is considering a diagnosis , of pulmonary 
infarction for a case with a low white blood count, the teacher 



if tti< 



lite blood count is consistent with 



might ^ask 

pulmonary infar(|Ib'n. /in fact a high white blood count is 
consistent with pulmonary infarction. 
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Identification Strategies 
ISls Aak for value of the dependent variable . . » 

* ■ 

If (1) a case has beer\ selected, and ^ " 
♦ ■ 

(2) the value of the dependent variable has not been specified; 
then (3) ask the student to identify- the value of the dependent 
var iable . 

, ISla: Suggest a value of th4 dependent variable 

■ ' (# ■ ' , • 

If (l)"a case has been selected, and 

(2) the student doesn't know the value of the dependent 
variable, 

then (3) suggest a possible value of the dependent variably for the 
Student to consider. ^ 

ISlb: Point out the value of the dependent variable 

If (1) a fjiaee has been selected, and ' ' 

(2) the student is mistaken about or doesn't know the value of 
the dependent' var iable,. , ' 

,theVi (3) tell the student the correct value of the dependent" 
variable. 

EXAMPLE (from Stevens a^nd Collins on the caused of rainfall) 

« T. Do you think it 'rainS much in Oregon? (ISl: Ask for value of 
the dependent variable) , * 



. T. Why do you tJjink iU^oesn't rain much in Oregcjn? (IS6: Ask 
* for' relevant factors) . ' ♦ * 

S. I'^n rw^t exactly sure just hypothesizing - it seems to me that 



the surrounding states have a rather dry climate, but I re 
don't know anything about the geography of Oregon. 



that 



T,. It does in faqt t!ain a lot in Oregon. (ISlb: Pbint out value- 

of dependent var iable rX^an you, guess what causes the rain 

' ' '. ' ^ . ■ • - ■ 

there? (IS6: Ask for relevant factors) 

IS2: Ask for the formulation of a ru4e - . 

— ' — — , — . i . » 

, If (1) one or. more factors -have been identified, * 

■■ * • :. 

then (2) agk how the tvalii^s of the factors are related to thp value 

of the dependent varviable . ' 

• . ■ .. . .. ' 

EXAMPLE , (from Anderson *on factors affecting temperature) 

T. Please try to be more precise ■ (e . g . , with re spec J to the effect 
of latitude. on terpperature) . Would you, for instange, siy that 
if you take any two places in the No#thern Hemisphere, the one 
furthest south ° has the colder , winter temperatures? 
(IS2ar, Suggest. t?e formulation of a rule) . ' 

r< , V ■ . ■ '. ■■■■ 

IS3: Ask for rthe f6rmulation of an alternative rVii^ 

! ^ ' ^ . : . : ^.1 f ,, ^ 

If (1) an incorrect rale has been specified relVting.the values of' 
one or more' factors with a. particular valueyof the dependent 
■\ • var i,able ' 
/^Hen (2) ask fqr th^ 'formulation of an, alternative rule 



EXAMPLE (from Anderson on factors affecting temperature) 

S, (In xeaponse to question under IS2 above) No I wouldn't sav 
that. 

4 ^ 

T. Whafe Would you say? . (IS3: Ask for the formulation of an 
alternative rulf^i) \ 

.IS4: Ask for sufficient factors ■ ' " ^ 

If (1) there are one or more sufficient factors that have not 4as*ff 
identified, 

the^rTH, ^sk the student to identify those factors, ^ y 

EXAMPLE . . 

Suppose a student has not identified irrigation or a means of 
obtaining enough water to .grow r ice* the teacher might ask "Is 
\ there any w^y to obtain enough water to grow rice other than 

■ from fainfall?". '• . . 



ISS'i Ask for necessary fa ctorfe 



r 



If (I) there are one or more necessary factors that have not been 
identified, ' , • 

then (2) ask the student to identify those factors. 



EXAMPLE 



/Suppose a student has not identified any factors that affect 
whether ! a . place has heavy /ainfall, a teacher , iii4ght ask "What 
is necessary t9 have hdavy rainfall in^a place?" 
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IS6: Ask for relev,ant factors 

y ' ; — ' 

If (1) ther^ are either neciessary or sufficient factors that have ' 
not been identified, 
t*bn .(2) ask the student for any relevant f-actors. ~ " ~ 

EXAMPLE (from Anderson on factors affecting temperature) 

T. Which Is likely to have the coldest winter days, Newfoundland, 
or Wontaaa? 0^S5': EiTtrapment into prediction based on 
insuf f icie-nt factors) ^ (In this case a secondary factor 
overrides a primary factor.) 

S. Newfoundland. 

T. Please give your reasons, for answering Newfoundland. 
(IS6: Ask for relevant factors) ' . ^ 

1S7: Ask for values of factors 

^ 1 — ' — — — — 

^Iff- (I) there a»re relevant factors that have been identified for a 
particular case, but 

(»2) the values of the faeitors have not been identified for that 
case, 

then (3) ask the student for pie values of the factors'. 
EXAMPLE (from Collins on /factors affecting grain, growing) ^ 

♦ 

. S. I suppose "^t he re are places, like Nigeria is pretty darn 
fertile. 

» 

t 

T. OK. It's fertile, but wha't - other qualities (I56: Ask for" 
relevant factors) Is the temperature warm or cold? 
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(IS5a: Suggest a necessary \actor1 (IS?! Ask for the value of 
, a factor) 

XS8: Ask for prior steps 



If (I) a particular step in a causal chain or procedure has been 
idfent if ied', and / 
(2) there are prior steps that have not been identified, 
then (3) ask the student to identify the prior steps. , • 



EXAMPLE (from Stevens an(^ Collins on causes of rainfall) 



* \ 



T. Where does the moisture in the air come from? (isaj>! Ask for 
prior' steps) < . • . 

S. Help. ^ ^ ' I . 

\ ' I • 

T. The moisture evaporates from the ocean. (IS8b: Point out 

prior stepsl ^hy do you think a lot of moisture evaporates? 
(IS8: (AskXfdr prior steps) 

9 : Ask for in tern|&<iate^steps • . 

If (I) two steps i\ a causal chain or procedure tha.t are not' 
adjacent have been identified, 
then (2) ask the ^udents^to identify the intermediate steps. 

EXAMPLE (from Stevens an^, Collins on causes of rainfall) 

^ •• 

S. When the moisture laden air reaches the mountains it is forced 
to rise and consequently the air cools? causing rainfall, no? 

T. Why does cooling cause rainfall? ^ (1S9: Ask for intermediate 
steps.) >■ 



ISlO}- A3k foe aubaegaept steps - - " - 

If (I) a particular step in a cau&al chain or procedure has been 

i^ient i f l<«d, and " .. 

(2) there are subseijuent steps that^ have not been identified, 
then (3) ask the student to .identify' the subsequent steps 

EXAMPLE: (from Anderson onT^o^ity of draft resistors) 

S. Yqu just can:' t have individuals deciding which laws they are 
going to obey. ' • • , . 

T. So, you would say the American tevolut ionar ies should have 
■followed' the ,law. (CSS9: Pipk a counterexample "fof an 
insufficient factor) 

S. Yes, I gue^s so. . ^ 

T.„ If they had obediently followed all the laws we might not have 
had the American revolution (ISlQa fx S&^st a subsequent step) 

IS 11: Agk^Jor si milar ities in fa ctor s b^tweVn similar cases 

It (1) two or more cases ha\>e beef! identified that have similar 
.vvalues on the dependent variable, 
then (2) ask the student tp identify any factors on which the cases 
have similar values, . " 

EXAMPLE (from-lfl/arman on morality of characters in Petqr Pan) 

T. What makes those characters good? (referring to Peter Pan, 
T,in|kerbell, and Wendy) (ISll; Ask for similarities in factors 
between similarr^cas(^s) | 
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IS12: 7V8k for drfferencea "in factors between similar 'cAaes 

If (1) two or more cases have been identified that have similar 

,.. . > * ' 

values on the dependent variable, 
then (2) ask the student tfll^dent if y any factors on which the cases 

. have different values. 
• ^ ■ ■ . 

- EXAMPLE •- 

Suppose that both Japan and Java' have been identified as 
producing rice, the^eacher could ask the student for any 
differences in factors between the two cases. in fact Japan is 
colc^er and much more mountainous. This, indicates that flat 
land and a tropical climate are not necessary factors. 

IS13: Ask for similarities in factors between different cases 

7 ■ ^ * — '- 

' If (1) two or more cases have been identified that have different 
values on t^ie de\^endent variable, 
then (2) ask the student to identify any factors on which the cases 
have similar values. 

EXAMPLE 

Suppose that Oregon has been identified as having a lot of 
rain, and Baja California as having littjj^ rai^, then the 
teacher might ask'Vhat factors they have in'' common. Since they 
ate both on the western coast of the continent, t^at means that 
that factor doesn^t determine the amount of rainfall. 

t 

/' 
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IS14; Ask for differencea in factors between different casm^ 

If •(!) two or more cases have been identified, that have different 
values on the dependent variable, J . 

then (2) ask the student to identify any factors on which the cases" 
, have different values. " ' 

EV(AMPLE (from Anderson on factors affecting temperature) 

S. Some other factor besides horth-South distance must also affect 
temperature . , . , 

T. Yes.' Right. What could this factor be? (IS5: Ask for 

necessary factors) 
S. I don^t have any Idea. ' 

T, Why 'don't you look at your map of North America. .Do you see 
any differences between Montana and Newfoundland? (IS14: Ask 
for differ\nce_s in factors between different cases) 

V 

« ♦ 

IS15: Ask f or.a-'case with a given value' on the dependent variable 

• . ' ' • ■■ ■ . V 

. If (1) there is no case currently being considered, and 

(2) there is a particular value of the dependent variable .t;o be 
considered,. .• \ . 

J then (3) ask the student to pick a (3ase that has that value on\ the 
dependent variA^le. 
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5»(AMPLE (from Collins on factors affectirxa grain growing) 

- T,.„.me.i:.fi -4a...l4ar.4Y do~"you- tlthlc"- rTce-- mTght be'^'' grownr"' 

(IS15;^ Ask for a case with a given value. on the dependent 
variaBle) ■ . 

\; / * 

S. Louisiana 

Ask for a case with given values on some factors 

If (1) there is no case currently be ing -considered, ' and ^ 
(2) there are particular values of some /siet of factors to be 
considered, \ 

then (3) ask the student for a case -that', has the given values on the 
set "of factors . ' ' * 



EXAMPLE 



Given a discussion . of rice growing, the teacher might ask a 
student if he knows a place where there\ is a lot of rainfall 
byt it is rather cold, (e.g., Oregon). ' ^ " 

' ' r y . .. 

Ask for a case with given values on some f^ctorj^ahd the" 
dependent variable 

If h.) there is no case currently being considered, and 

.(2) t^ere is some pairing. of values on particular factors an( 
on the dependent variable to be considered. 

> . . ^ 

then (3) ask. the s.tudent for a case th/Jt has the given values on the 
factors and on the "dependent vaVtable. 
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EXAMPLE 



■GTven i discussion of ric^ growing, the tether might ask a 
student if he knows a place where fcfi^TT^ a lot of . rainfall, 
but no rice is grown (e.g., oWgon) 




■A 



Evaluation Strategies 

' — — - — — 

^S^* Ask if the value of the dependent variable is correct or 
incorrect ' 

- • ^ " V 

t 

If (1) a value has-been suggested for the depen'(5ent variable in a 

• ') 
particular case, / . 

then (2) ask the student if that value is correct or incorrect. 

EXAMPLE (from Collins on factors affecting gre^in growing) 

I 

T. What do you think' they live on in West Africa? (ISl: Ask^ foi 
valu^of the dependent variable) 

« 

S. I guess they grow some kind of grain in West Africa. 

/ 

What kind is most likely? (isi: Ask for value of the dependent 
variable) 

S. Wheat. > 
T. You think wheat is the most likely grain? (ESI: Ask if the 
value of the dependent variable is correct or not)' 

ES2: Ask if a rule is correct or incorrect 

If (1) a rule has been suggested relating a set of factors to the 
dependerft variable, 
then (2) ask the studenV if the rule is correct or incorrect.* 

EXAMPLE (from Warman on who can play with blocks) 

T. How about if we had boys coUld play wiijh everything but blocks? 
(CSS13; construct a hypothetical^ oa6e for insufficient 
factors) (ES2: Ask if rule is correct or incorrect) '(Warman- 



i 



f 



treats fairness as the dependent variable, and here suggests a 
^ ^ 1« de^r i ved by cons ttucti rig a " hypo the tlcal case " for 
insufficient factors,) ^ 

ES3: Ask if a rule is the same as or different from another rule 

If (1) a rlile ha^ been suggested which appears similar to anot^her 
rule, 

then (2) ask if the rule is the same as or different from the other 
rule . • ^ 

EXAMPLE (from Warman on who can play with blocks) 

51. I've got a good idea. Everybody play with blocks.. 

T. What do you think about that? (ES2:' Ask if a rule is correct 
or incorrect) 

52. Rats. ^ ^ • 

T. Isn't tz'hat the rule we have now? (ES3: Ask if a rule is the 
same or different from another rule) 

V 

ES4: Ask if factors -are sufficient or insufficient 

If (1) one or more factors have been identified with respect to a 
particular value of the dependent variable, 

r 

then (2) ask the student if the factors are sufficient or 
insufficient to aetermilie the value of the dependent -variable. 

EXAMPLE (from Swets and Feurzeig on identifying letters) \ 



T. start when* ready (The student must guess a letter , from ita 
features) 
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S. Ciicvea?* • * 

T. One, 

> . • • 

S . Loose ends? ' ' ' . 

T; Two. 

S . Obliques? . - v ' 

T. Zero. . ♦ - 

S . O. - • 

T. You don't have enough information yet to get t^je right answer. 
(ES4b: Pointy out that factors are insuf f icient) , How do you 
know it isn't J, for example? (ISia: Suggest a value of the 
dependent variable) ■ * ' * 



J : 



1 

ES 5 : Ask if factors are necessary or unnecessary ' 

If (1) one or more factors have been identi-fred with respect to. a 

' ' >' ■ . ■ ■ ^ • . 

particular value, of the dependent variable, ' • 

then (2) ask the student if- the factors are necessary or unnecessary • 

to determine the valpe of the dependent variable. * 

EXAMPLE ' ■ ,' 

- Suppose a student suggests that places with a lot .6f rain caji 
grow rice,, the teacher might ask "D© you have, to have a lot o,f ' 
.rain in order to grow rice?" ' 

ES6: Ask if factors ar6 relevant or irrelevant ^ 



If (1) one or more factors have been identif ied, with respect to 
particular valae of the dependent variable. 



a 
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then, 12Ua.5K^thfi s.tu4efht--4*~the-"-factars--arg - relevant or ir i^^e van t 

, - to the value of the depetiaeht variable. 

■ \ ' . \ ■■■ . . 

EXAMPLE (from Warman on who 'can play with blocks) ^ 



S. HOW about all the boys take all the blocks and put, them outside 

■ , • ■ ■ . % 
and the blocks stay 9utside „the building. .v 

T- So Kave the blocks outside the building. (Restate rule) Then 
do we still have the problem? (ES6 : Ask if a factor is 
relevant- or^ irrelevant) (Warman is ask ing ' (%Jie th^er having^ the 
.blocks' outside' is relevant to fairness.) 

Ask; if; the values cf fa(^tora ar e correct or incorrect 

. ' ^ V ' ' 

'.rf ' (1) the values of one or mbre factors have> been identified with 

>^ xespect-to a particular value of the dependent variable, 

thdn. ('2) .ask the student if the values of the ifadtors are correct or 

• - incotfeVt with, respect to the value of the dependent variable. 

EXAMPLE Cfrom fewets and Feurzeig on medical laiagnosis) 

••T. In 'that .case I'd like to talk about; ;viral 'pneumonia." 
(15'la-t Suggest a value of the dependent variable) The 
tachycardia, high WBC, elevated respiratory rate,' shaking 
chills, bloody sputum, and severe pleural, pain all lend weight 
to that {diagnosis, - right?' MES7: Ask if the v^alues of factors 
are correct or incorrect) 



If (1) there are two .steps identified ■ in a causal chain or 
procedure, 

then (2) ask the student i-^ one step. is pr^or \o the other step or 



not.. 

EXAMPLE . < 



In discussing wh^t cautees rainfall, ^the student might mention 
the air pooling and ris;ing'. the teacher might then ask the 
student if the air cools before it rises. 



4 
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V 

Ask if a step is an intermediate step ' 



, If (1) a given step in a, causal chain or "procedure has been 
identified with respect to twb other steps, ■ V , 

then (2) ask th? , student if the step is intermediate between the 
Other two steps. 

EXAMPLE . ' 

• ' *". • 

Suppose a student is learning 'about evaporation p^rocesses, the 
teacher 'might ask- whether clouds <.form after vaporization takes 
.place, but before condensatiorf oceu.rs. 'cloud formation is in 
fact cause^ by; condensa t iofi . ♦ • 

ESlOj Ask if a 9tep is a subsequent step • 

If (1) a giv.^n step in . 'a causal chain or procedure ha^s been 
identified with' ;:espect to another step, 



* 1 

then (2) laakr the student if the step is subsequent to the other 



step. 



EXAMPLE 



Suppose a student is l^earnirvg . the distributive/ law in 
arithmetic (as in one of the' Anderson dialogues), Ahen with 
respect, to the problem 7 X 12 + 3 X 12, = ?, tihe feacher might 
ask if .you multiply by the 12 after adding' th^ 7- 



ESll: Ask if si'mi.lar cases are ^e same on given factors 

. . . ' i . 

If (1) two or -more, cases have been identified that have the same 
value on the dependent var'iabie, and 

(2) there are one or more factors tor which the cases have^he 
same values, ' 




1^^ 



thea (3) ask^the student if the cases htave , the same ^ or different 
ilu'es'on th-e given factors. - . > . 



EXAMPLE 




Suppose the st^udent is ,learning^about .the c^ses of ^rainfall, 
. ♦ - * 

and the student notices tha.t Baja California and Northern Chile 

have little rainfall, the teScher might ask if they have the 

same latitude (which they do). 

.ES12: Ask if similar cases are d ifferent on given factors 

r- — ^ 2-^^.^ ^ • ^ 

If '(1) two or more cases have^been identified that have the same 



rvalue on the dependent variable, and 



1 

\ 
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(?) there are one or more factors for which the cases have 
different values, 

then (3) ask tjhe student if the cases have ,the same or different 

s 

values on the given factors. 



4 



EXArtPLJE * , . ^ 

1 * 

Suppose a student has identified the .Amazon and Oregon as 
having a^ lot 'bf rainfall, then the teacher could ask if they 
have the same or different values pn latitude and altitude 

'-^v(they differ on both). " iS? 

■ . ■ •• . ' '^'^ • 

.^2^3: Ask if dissimilar cases are the same on, given factors 

* > • ■ . 

If af two or more cases have been Identified that have different 

values on the dependent variable, and 

J 

. (2) there ^re one or more factors for which the cases have the 

same values, , * ' * > 

then (3) ask the student if the cases have the same or different 

« values on^he given factors. • ' 

*(»■,- 

•EXAMPLE (from Ander*son 'on morality df draft resistors) 



T. You are saying that what the draft resistors, did" was wronq 
^ ^Decause they jproke the law. The American revolutionaries broke 
the laws tdo. (CSS9: Pick a counterexample for ^an 
ii%suf f tcient factor) ' (ES13^b: Point out -that two dissimilar 
< cases are the Same on a given factor) Therefore to be 

consistent, you wouid have to say that what they did was' wronq 
- (ISla: Suggest a value of the dependent variable) 
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ABk if disa lTtilXar caaea'are different o n given factors ^ 

If (1) two or more cases have be«.n identified that have dif.fei^ent 
values on the dependeivt variable, and ^ 

(2)\ there are one or .more factors fo^r which the cases have 
^ different values> 

♦ 

then (3) ~ask the student if the cases have t^e s^e or different 

values on the. given factors. • * • 

|r. ' - > . - 

EXAMPIiE . r ' , ' 

Suppose a student has identified Sumati»a and Java as having 

different ^population densities, the teacheY might ask if they 
have the same terrain. * ' 



r 
4 
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San Diego,. CA 92152 
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Navy 



1 CAPT Paul Nelson. USN ^ ' 
Chief. Medical! Services Corps. 
Qodej ' 
Bureau of ried icir\e^ 4 Surgery 
U. S. ^Department of the Navy 
V^shington. DC 20372 

1 -Library 

Navy Personnel R4D Center 
San Diego. CA 92152 

6 . Commanding Officer 

Naval Resergrch Laboratory 
Code 2627 ^ c ' 

Washington. DC 20390 ^ 

J JOHN OLSEH- . 

CHIEF OF fJAVAL EDUCATION & 

TRAINING silPPORT 
PENSACOLA,' PL 32509 

1 Psychologist ' 

ONR Branch Office ^ 
VM95 Slimmer Street 
■ Boston , MA -pSSin ^ 

1 Psychologist • 
ONR Branch Office 
S36 ^. Clark Street 
Chicago, IL 60605 

1 Office of Naval Research 
Code -200 * 
Arlington, VA 22217 

♦ 

1 Office of Naval 'Research 
Code ^y! 

800 N.. buincy SStreet 
Arlington, "vj 22217 " 

/ 

5 Personnel Tt Training Res.earch 
(Code/i<59) ■ ' • , 

Office (\f Maval Research 
Arlington, VA 22217 • • 

1 Psycho log laT^ » 
OFFICE/O^ NAVAL RESEARCH BllANCH 
♦ 2^3-^D\MARYLEfiONE ROAD • 
LONDON, m, 15TH ENGLAND 



rams 



,1 Psychologist 

ONR Branch Office 
1030' East Green Street 
'Pasadena, CA , 91 1i31 . 

1 Scientific Director' < 

Office of. Naval Research ^ 
Scientific liaison Group/Tokyo 
American Ernbdasy 
' ' A^'Q San Francisco, 'C"A 96503 

^^ Office of the Chief of Naval Operations 
Research, Development, and Studios Branc 

<OP-102) 
Ifl/ashington , DC 20360 

1 Scientific Advisor to the Chief of. 

Naval Personnel (Perd-^O?;^) • 
Naval Bureali of Personnel , 
Room MMld, Arlington Annex 
Washington, DC 20370 

1 LT Frank C. Petha, t15C,*U3MR (Ph.D) 

Code L51 ' 
Naval Aerospace Medica*! Research Laborat 
. tensacola, FL 3250« 

1 DR. RICHARD A. POUAK 

ACADEMIC X;OMPUTING CENTER ^ 
■ U.S.' NAVAL ACADEMY / ' 

ANNAPOLIS, m 21402 



Rojjfr Vi. Remington, Ph.D 
Code L52 

nAmrl 

Pensacola,. FL 32508 



Mr. Arnold Rubenstein 
Naval Personnel Support Technology. 
Naval riaterial Command' (03T244) ' 
^R6'om""TWV'Xrystal""Tlazci"F5 
2221 Jefferson Davis Highway 
Arlington, VA 2C360 

Dr . Worth Scanlond 

Chief of Naval Education apd Training 
Code N-5 . ' " ' . 
NAS,. Perlsucoia, FL '32503 
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A. A. SJOHOLM * ^ ' 
TECH. SUPPORT, CODE 201 - 
'NAVY PERSONNEL RS D CENTER 
, . ■ SAN DIfGO, CA 92152 

1 ^ Mr . Robert. Sin.ith 

Office of Chief of Naval Operations 
OP-037E ^ •. , 

Wdshington, DC 20350 

I 

^, 1 Dr . Alfred F. Snode • 

Trainirjft Analysis & Evaluation Group 

Dept. of the Navy 
; Orldndo, FL 32813 . » • 

1, Dr . Richard ^orensen 

Navy, Personnel R&D Center . 
. , . San Diego , CA 9215? 

1. 'CDR Chorl<>s J. Theisen, JR. M.SC, USN 
,Head Hunan FactorjlMfcjii nee ring Di v . 
Naval AiV 'Dev<ol0^llW|I^Certfeer• 
W<^^mins^er PA 1897^1 

V 1' \> ' Giiry Thomson 

Nav.il Ocean Systems Center 
Code 7132 



in Diego. OA 92152 
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ArmV 



1 '^Technical Director » ' . 

U. S.»Army Rese^irch Institute for tht 

Behavioral • tind Social Sciences 
5001 Eisenhower Avenue 
Alexdndria , VA 22333 

\ 

1 HQ USARF.UE A //th Army • . • " 

ODCSOPS* ^ 
U3AAREUE Director of GED • 
APO New York 09'l-03 

1 LCOL Gary Bloedorn , 

Training Effectiveness Anal^fis Division 
his Army T|?ADOC Systems Ana^is- Activity 
White "Sands Missile Ran^e , Nl'*l 83002 * 

1 DR. RALPH DUSEK 

, U.S. ARMY RESEARCH INSTiTUIE 
5001 EISENHOWER AVENUE 
'ALEXANDRIA^ VA 22333 

1 Col. Frank Hart, Director \ ' 

' Training Development Institute 
ATTNG-TDI 

* Ft. Eusti>s, VA 23601 ( , ' 

1 Dr.. Ed Jol nson 

Army. Research Institute -. 
5001 Eisenhower Blvd., 
Alexandria, V/f 22333 , « 

1 Dr. Mich.iel Kaplan 

U.S. ARMY RESEARCH INSTITUTE 
5001 Ei:3ENH0UER AVENUE 
ALEXANDRIA, VA 22i33 

1 or. Milton G. Katz 

Ind iv id lRjI Training &-Skil4 
Evaluatiori*Te'chnic^jl Area - 
U.S. Army Research Institut* 

5001 Eisenhower" Avenue ' — - • 

Alexandria, VA 22333 ' 

1 . Dr . Beatrice -J. %ir.r '.. 

Army Research Institute . (^'PER^-(^j<) , 

5001 Eisenhower Avenue \" 

Alexandria, VA 22333 
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Air Torce 



Dr. Harold F. O'Neil, Jr. 
ATTN; PERI-OK 
5001 EISENHOWER AVENUE 
ALEXANDRIA, VA ^ 22333 ' ^* 

LTCOL''MlM:haer T. Plunmer , 
Organizational ^Effectiveness Division 
Office o^ the Deputy Chief of Staff 

for Personnel' 
Department of the Army 
Wasfiington , DC 20301 ' 

Dr. Robert Sasmor 

'U. S. Army Research Institute^ for. the 

Behavioral and Social Science^ 
5001 Eisenhower Avenue 
Alexandri^, VA 22^33 ^ ' 

\ 

Faculty Development Division 
Headquarters Sheppard Technical 

Training Center (ATC) ' 
Sheppard AFB, TX 76311 ' 

Director. Training Deve^loprai>nt 
U.S. *Army Administration Center * 
ATTN:' Dr. tSherVill 
Ft. Benjamin HarMson, IN 46218 
% . ' 

Dr . Joseph Ward 

U.S. Army Research Institute' ^ 
5001 Eisenhower 'Avenue 
Alexundria, VA 22333 



V 



Air University Library 
AUL/LSE 76/4^3' 
Maxwell AFB, AL 361 12 ' 

^ ■ 

DR. E. COTTERMAN ' 
AFHRL/ASR / 

WRIGHT PATTERSON AFD x \ 
OHIO ,45'! 33 

DR. G. A. ECK3TRAND 
AFHRL/AS 

WRIGHT-PATTEflSON AFB,.OH 45^33 
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Dr. "Genevieve Heddcd 
Pfogriim Manag^er ' ^ 
Li fe Sciences Directorate 
AFOSR ' ' 

Boiling AFB, _DC 20332 

CDR'. MERCER 
CNET LIAISON OFFICER 
AFHfiL/FLYING TRAINING DIV. 
WILLIAMS AFB; AZ 85224 

•Dr. Donald E. Meyer 
U.S. Air Force 
ATC/XPTD 

Randolph AFP, TX 78l4y 

Dr. Ross _L. Morgan (AFHRL/ASR) 

.VJright -t'dtterson AFB 

Ohio 45433 - " ' 



Dr. Marty Rockway (AFHHL/TT) 
Lo.wry AFB 
Colorado 80230 



^1 



Jack A. Thorpe, Capt, USAF 
Program Manager ' • ' 
Life S(fLences Directorate 
AFOSR 

Boiling AFB*, DC 20332 ' 

Brian K. Waters, LCOL, U3/fF 
Air University 
riaxwell "AFB • , * 

Montgomery, AL 3|1 12 
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"^•"^"63 • ^ . ; 'coaatGuard 

t ■ H.*^'lliarn Greenup . - 1 • Mr. Richard Lanterrntin' 

Education Advisor (EO3I) PSYCHOLOGICAL RESEARCH (G-P-1/62) 

Ed.ucation Center, .MCDEC • - U.S. COAST GUARD HQ 

Quantic.. .VA 2213'4 WASHINGTON, DC 20590 ■ 

1 • DR.- A.L. SLAFKOSiKY 

SCIENTIFIC ADVISOR (CODE RD-1) • . , 

HQ,. U'.S. MARINE CORPS 
WASHINGTON, DC 20380 



\ 




t 



r 

\ 

V 



BB4N/fb\Una' Qctt^ber 10, 1979 



Other DoD 



Civil 00^,1 



1 Dr . Stephen Andriole 

ADVANCED RESEARCH PROJECTS AGENCY 
• MIOO WILSON BLVD. . 
ARLINGTON, VA' 22209 

» 

12 Defense Docu^ent^tion Center ' ""^ 

Cameron Station, Bldg. 5 ' : 11 
Alexandria, VA 22314 ' ' . 
Attn: TC ' . ' • ^ 

1 Dr. Dexter Fletcher 

ADVANCED RESEARCH PROJECTS AGENCY^" 1 
IMOO WILSON BLVD. 
ARLINGTON, VA 22209 

1 Military Assistant for Training and" 
. Personnel Technology 
Office of the Under Secretary of Defensd ,1 

for Research & Engineering 
Room 3D129, The Pentag.on- \ . ' 
Washington, DC 20301 ' /* •' 



' Dr. ^usai) Chipmari 
Basic sMlls Progt-din 
•National ^Institute of ,£ducution 
1200 19 6h 'street NW 
Washingto^n, DC 20208 ' .: 

- -l 

fir. James M. Ferstl 
Bureau of Training 
U.S. CiviX, Service Commission 
Washington', D.C. 20415 • 

Dr. Joseph I. Lipaon 
Division of Science Education 
Room ^W-6j8 ^ 
Na4:.ioncil iScience Foundation 
Washington, DC 20550 

*Dr • Jdhn ^Mays 

National Institute of Education 
1200 19th' Street NW 
Washington. DC 20208 * r 

( 

4 William J. McL^uriY) 

Rm, 301, Internal Revenue Service 
-■2221 Jefferson Davis HiBiiway" 
Arlington , ,VA "22202 • 

1 Dr . Arthur Helmed 

National Intitute of Education 
■ 1200 19th Street NW 
.Washington., DC 20208 

1, Dr. Andrew R, Molnar 

Science Education Dev . 
and Research 
' National Science Foundation 

Washington , DC 20550 

1 Dr ? H. Walii^ce Sinaiko 

Proyrat;! Director ' , 

Manpdwi?r /Research and Advisory Z 

Smithsonian Institution 
r J01 'Nor^th Pitt Stree.t^^ 

AlexfinJria. VA 223 V4 
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Non Govt 



1 Dr . Thomas G. Sticht 
BaSio Skills Program ; 
National Institute af . Educption 
','1200 19th Street NM 
• Washington, DC 20208 

1 Dr . Frank Wi throw ' ; 

U, S. Oftfice of Education 
'100 ^th/^Str^t SW^ 
Wash ipg ton, DC 20202.-^ 

1 br . Joseph L. Young, Director ' 
Memory & Cognitive ^Processes 
National SGlence' Foundation 
■ Washington , DC 20550, v 



IN- 



Dr. Earl A. Ai;uisi . 
HQ, AF.HRL (AFSC) , •' ' ' 

Dndoks AFB, TX 782 j5 '• 

t 

Or . ^ohn R. Andt^rson ' ♦ 

Department of -Psychology 
Carnegie h^llon»! Univei;sity > - 
Pittsburgh; PA 15213 

», 

Dr . , John Annett 
Department /^f Psychology 
University' of Warwick 
Coventry cy^ 7AL * ' ' . 

ENGLAND ' .. , , ' ] 

i 

DR. ^IICHAEL- ATWOOD , ' ' \ i ' 

SCIENCE APPLICATIONS INSTITUTE " • \' 
, ^0 DENVER TECh. CENTER WEST 
7915 E. PRENTICE AVENUE 

ENGLEWOOD, "cO 80110 . _ 

1 1 psychological 'research urtit . ' ' 

Dept. of Defense (Army Office) i 
Catapbell Park Offices 

Canberra ACT 2600. Australia ' " i 

1 Dr. R.-^. Avner • ' .• v,^^ \ 

/" University of Illinois , j ' \ 

Computer-Based Educational Research Li^b \ 
Urbana^IL. 61801 



Dr . Alan Baddeley . ^ 

Medical Rese^r^ch Council 

Applied rPsyohology^nit* 
15 Chaucer Road . 
Cambridge CB2 2EF 
ENGLAND 




Dr. Patricia Bagglett 
Department of Psychology 
University of Denver 
University Park 
Denver, CO 80208 

f^3. Carole A'. Barley ^ 
Minnesota Kduoational Computing 

Consortium 
^520 Broa'.iw;iy Drive 
St. PjuI, MN 55113 
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Mr |\v.ro^i Briri" \ 
f)v.y!}frli\\ont of Computer Scienct 
'Stcirifdfd Unlver^ii^^- 
Stanford.- CA 

nr. John Dergan 
School' of FduccitlonJ 
UnivorsLi.y oT Arizona' 
Tu3Gori/AZ 8b721 

TiVu Nicholas A; Bone, 
Dept. of Psychology. 
SucrumcTito Stjtq College 
. 'lOt) Jciy :^roeL • 
rj^icramentp , CA 95319 . 

Dr. Lylo' Uoiarne 
Dep.jrtnient of Psychology 
University of Colorado 
nould^ , CO 80 302 

% Dr. Kenneth rov;l(^i5 • 
»^lnstituto for Information Sciences 

University of California at San Dicg 

La JoUa, CA 9?0 37 

• \ 

Dr , John Brjckett ^ 
v'iofTeqh 

'160 Totten Pond Road . 
Walth^m, MA 0.'i15'4 ' 

Dr. John 1">, Prown 
.XEn()X P?Io Alto Research Center 
]}^^, Coyote Ro id 
Palo Alto , CA 9't30'l 

Dr. DruL'P Puchanan 
Department of Com'ptJter Science 
• Stanford University 
'Stanfbrd., CA 9430^^-^^ " 

DR. C, VICTOR BUNDERSOM 
, WICAT INC. 

UMIVKRSITY' PLAZA', SUITE ,10 
1 ir)f> so. STATE ST. . 
OREfl, UT Mii0'37 



■Hon Govt 



Dr'. Anthony Cancel li 
school of Education 
University of Arizsna 
Tuscon. AZ 8^721 

Charles Myers LLbrdry 
^Li V Incstone. House . 
Liv ingstone RoucJ * 
Sferritford 
London 2LJ 
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ENGLANI> 




Dr . Wllli'jn CHase ^ 
Departnrent of Psychology 
Cjrnegie Mellon University 
^'ItLsburgh, PAJ5213 

• '* 
Dr. Mich] 

Learning \R &U) ]Cc)(iiter^ 
Univevrsi t^^^^rT^pTttsburgh 
3939 0'bl^rcr5treet 
PlttsbuLh, PA 15213^ 

Dr. John Ch^lTTflrii 
■ Litton-Myllgnics 
Box 12«6 

SprhiGfK'ld, VA -22151 ' * 

" Dr • V/illiain Cl^jnct^y ^ 
Deptir tr.ii'nt o f ;Comi^uter I'ciqnoe / ' 
ot anf o rd Un i V er si t y 

Dr. Meredith Crawford 

Depar.tnent of En^iinter in,i adininistri.tidn 
George WasfungLori University ^ 
Suite 805 • ' ' 

2101 L Street N. W. , ■ • 

V/ashington, DC 20037 • 

Mr < Ken Cross 

Anacapa Science^, Inc. • 

-P.d. Draw'.sr Q 

Santa Barbara , CA 93102 

Dr'. Hubert Dreyfus i 
. DeVartntnt of Philosophy •'• 
UnKrersity of Call^rn|.i. 
BoPkely, CA 9't720 , .> 
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1- . *ERI(: Facility-Acquisitions 
il83< Rugby* Avenue 
Detheada, HD .2001'! 

1 .Dr. A. J.- E3chen()renifer 
P«pt. E^22, Dldii. loj 
McDonnell Douglas Astronautics Co. 
P.O.Box ^T6 ' '■ 

St . Louis', MC 6^166 ' *■ . 

1 riAJOR i: N. EVONIC 
^ CANADIAN FORCES PtRS. APPLIED RESEARCH^. 
: 1107 AVENUE '^6^D 

TORONTO.,. ONTARIO, CANADA 

1 Dr • Ed F'elgenbaun J 

Department of Cornpiter Science 
Stanford University ' 
, Stanford . CA <)^ i05 /' . * 

1 Mr • Wailac^ FeurzeiK ' 
^ Doll BeranV A Newman, Inc. 
•* ' 50 Mo 111 ton Gt . . 
Carnbridiie . MA >021 38 

1 Dr. ViQtor Fields / 
Dept. of Psychology 
Montgomery Co H ego 
^ Rockville. MD 20350 

1 Dr. John R. Frederlkiien 
Bolt Berohek ^ Newman 
50 (doulton Street 
Cam Br id go, 02\ifi 

I ^ ^■ 

V- Dr. R. Fdward Goiselman 
Department of Psychology 
University of CaUTornia 
• Los Angeles ^ CA gOOPU 

\1 Dr. Vernon's. Qeriaoh 
^ College of Education 

1'»^ P.iyno Pldg. B ' 

Arizorui State University 

TenFHi, AZ 85281 



Non Govt 



DR. R013ERT GLASER 
LRDC 

UNIVERSITY OF PITt^DURGH 
39:-i9 O'HARA STREET 
PITTSBURGH, PA 15213 ' 

m 

Dr Ira GeddsUin' ^ 
XEROX Palo Alto ,Re3ear6h Center 
J333 Coyote Road 
Palo Alto, CA 9^60i\ 



DR. JAMES G.. GRBENO 
LRDC 

•UNIVERSITY OF PITTSBURGH. 
3939 O'HARA STREET ' 
P.ITTSBURGH, PA 15213 



V 



.1 Dr Ron Hunibleton 

School of Educiition- 
• University of flassechusetts 
Amherst , MA 01002 ' 

1 Dr., Darb^'ara Hayes-Roth 
Th'e Rijhd Corporation 
17QD'Main Street 
Santa .Monica , CA 90^^06' 

1 Dr. Frederick Hayes-Roth J 
The Ran^ Corporation • 
1700 Main Street 
Santa Monica, CA 9OHOO 

1 Dr. Dustiri H. Houston 
\Jicat, Inc. 

Box 936 ' ' . 

Orein, UT 8*^057 

1 Library , 

MumRRO/Western Division 
27857 Berwick Drii/e 
Carinel , Cl\ 53921 

1 Dr . • Eyrl'Hynt' 

Dept.. of Pfiyi;holoi.;y 
Universityi of V. jshington 
Seattle, V;K.9u105 ' 
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Non Govt 



\ 



1 DR. LAWRENCE B. JOHNSON ' 

LAWRENCJ- JOHNSON 4 ASSOC., INC. 
^SUITE 502 
"2001 S STRIEET NW 
WASHINGTON, DC 20009 ' .. * 

1 Dr. Wilson A. Judcj 
McDonnel l-Douglci^ 

Astron:iutic3 Co. East 
Lowry AFB 
Denver, CO ^0230 

1 Dr. Arnold F. Kanarick 
Honoywel 1 , Inc . 
2600 Rid^iew.jy Pkwy 
Minneapolis, flN b5'l13 

1 Dr. Walter Klntsch 

Department of Psychology 
University of Colorado 
Boulder., CO 8^302 

* ' " . ■ 

1 Js^' Dnvid Kior^is 

D^pdrtttiefit of Psychology 
Univt-rslty of Arizona, 
Tuscon, AZ 8S721 

1 Dr. Stephen Kosslyrf 
Hcjrvard University 
Department of Psychology • 
33 Kirklnnd Street 
Cambridt^e, flA C,?138 

1 Mr. Marl in Kroger 
' 1 1 17 V^a Gol^ti r l 

Paloi Verd^'i EstnteS, (^A 90274 

1 LCOL. C.R.J. LAFLEOR 

PERStoHNEL APPLIED RESEARCH 
NATIONAL DEFENSE HQS 
ini COLONEL BY DRIVE 
OTTAWA, CANADA K1A OK? 

Dr. Jill Ljrkln 
Department of Psychology 
C^jrnegle Mellon University 
Pittsburgh,' PA 15213 
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Dr . Alan Lesgold 

Learning R.'itD Centern ' ' • 

University of Pittsburgh • 
Pittsburgh,* PA • 15260 

Dr . Robert R. Mackie 

Human Factor ^^^R^search , Inc. 

6780 Cortona Drive^ 

Santa Barbara Researc[i Pk . 

GolGta , CA 93017 

Dr . Mark Millpr . ^. 

Systems and Ihforin&tlonSclerices Loborat 

Central Research Laboratories 

TEXAS INSTRUMENTS, INC. " ( 

Mall Station 5 

Post-. Office Box 5936 . * ' ' * 

Dai lad, TX 75222 . 

Dr . Rljchard B. Ml ;i ward ' , • 

Dept. of Psychology 
tlunter Lab . 
Brown University 
Providence, RI» 82912 



Dr . :5tuart Hilner 
Departnerft of Education 
George Hason University 
4400 Fairfax Drive 
fairfijx , VA 22030 



I 



Richard T. tlowday 

College of Cusiness Administration ^ 
Univ(Tsity of Oregon 
Euuene, OR 9740 j 

Dr . Alien Munro 
Univ. of So. California 
Behavioral Technology Laba 
3717 oouth Hope Street 
Los Angeles, OA 90007 

Dr . Donald A Norman 
Dept. of Psychology C-009 
Univ. of California, Sc^n Dlcgo 
La JoUa, CA 92093 
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Non Govt 



Institute for Defenle An^lys^,^ 
^100 Army Nav,y Drive " * " 

Arlin^tpx). VA 22202 

Dr. Seymour A. Papert 
Massachusetts Ihstit^ute of Technology 
Artifiqi^l Intelligence Lab / 
5'I5 Tec^hnolbgy Square ' ' ' 

' Cambridge, MA 02139 

Dr. Japves A. Paulaan 
"Portland State University 
P.O. Box 751 , • » 
Portland, OR 9720? ^ 

Mr. A. J. Peach, President, 
Eclectech Associates', Inc. 
P. vO." Box 178 
N. ' Stoning ton , CT ' 06359 

MR. LUIGI PETRULLO ^ " - 

24 j1 N.. EDGEWOOD STREET 
ARLINGTON, VA . 22,20 V 

Dr .• Barbara Pf lanz 
Department of German 
University of Redlands 
Red lands, CA 92J73 

DR. PETER POLSON 
DEPT. OF PSYCHOLOGY ^ ' 

UNIVERSITY OF COLORADO 
BOULDER, CO 30302 

DR. DIANE M. Ri\MSEY-KLEE 

R-K. RESEARCH SYSTEM DESIGN . ^ , 

3947. RIDGEMONT DRIVE 

MALIBU. CA 90?65 

Dr . Peter B. Read 

Social Science Research Council 

605 ThlFd Avenue 

New York, NY 10016 



X 1 




I* Dr ., Mark D. Rec^kase ' 

Educatiortai Psychology Dept» 
Ij^iversity of MLssour i-Columbiy 
12 Hill Hill ^ 
Cplumbic), MO 65201 

» 

I Dr. Fred Reif » . *: 

SESAME 

-c/o Physics , Bej^artment 
• University of California 

Berkely,. CA 91720 ^ ■ 

Dr . Ernst Z. RoLhko|f ' 
Bell Laboratories 
600 Mountain Avenue 
Murray Hill, NJ 0797'! * 

i Dr.. David Rumelhart 

\ Center Voik Human Information Processing 
^ Univ. of California; San Diego ' 
, La Jolla, CA 92093 

« 

Dr. Allen Schgenfeld 
Department of Mathematics 
Hamilton College 
Clinton, NY 13323 

DR. ROBERT J. SEIDEL 
INSTRU^X^ONAL TECHNOLOGY GROUP ' 

HUHRR^i \ :.: 

30n N. WASHINGTOII ST. ( 
ALEXANDRIA, VA 22311 ^ ' 

Dr<^Roberb Smith ' 
.Depaptmtnt of Computer Sciehco 
Rutgers \un iversi ty • . 

New Brunjswick, NJ 03903 

Dr . Richard Snow 
School of Education 
Stanford University 
Stanford, CA 9M305 



Dr. Robert Sternberg 
Dept. of Psychology 
Yale University 
Box IIA; Yale Station 
New Haven , CT 06!j20 
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OR. ALBKRT STfeiVENS 

I'OLT HprHAMEK '& NEWMAN, IMC . ' 

5n MOULTON STREKT . 

CAMBRIDGE, MA OP138 ^ ' 

Mr , Will lam Stobie * - 
He Donnel 1-Doygl as 

A^iLronaut", ics Co . 
P. 0. Box 30?OU " 
-Chlco, CA .9S926 " " 

t 

Mr . n. J. Sullivan .• • 
. o/o Canyon Rti^earc^h Group, Inc. 
741 Lake field Road 
West'lake -Vil lage , CA 91351 

DR. PATRICK SUPRES ' 
* INStmiTE fOR M/VJHEMATICAL STUDIES IN 

. - THE SOCIAL- SCIENCES 
STANfORD UNIVERSITY 0 ^ 
STANFORD, CA.. 9 '1 305 ' ) 

Dr, Kikuml Tat3u6k;i 
.Computed Based^Educifution Resey/ch. , 

L/ibor^ to ry * 
2S2 Enginserlng Research Labofatory . 
University of llHnois 
Urbana, IL 61 "01 

Or . John -Thomaa 

jnn^ ThQrn.je' J, yatson Research Center^ 

P,o\ POX-31R ' 

Yorktowri HeUhts, NY 10^98 
^ Y^ * 
. OR. PERRY THORHDYKE ' 
' THE RAND. CORPORATION . • " 

17no MAIN STREET 

SANTA^ MONICA, CA 90106 
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Non Gov't 



Dr . DoufilAis Towne ^* 
Univ . of po . Carifor'ntsJ 
Beharvieral Technology Labs 
37>7 South Hope Street ' 
Los Angeles, CA 90007 

Dr . J. (rUhlaner^ 
Ptfrceptronics , Ipc. 
6271 Variel Avenue 
Woodland Hills, .CA 9136') ' 

Dr , Phyllis Weaver 
Graduate School of Education 
Harvard University 
200 Larsen Hall, Appian Way 
Cambridge, MA 02138 



Dr. David J. Weiss ' 
N660 Elliott Hall ^ 
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